Jun 02 02:45:19.932411 ip-10-0-138-19 systemd[1]: Starting Kubernetes Kubelet... Jun 02 02:45:20.277986 ip-10-0-138-19 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 02:45:20.277986 ip-10-0-138-19 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 02:45:20.277986 ip-10-0-138-19 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 02:45:20.277986 ip-10-0-138-19 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 02:45:20.277986 ip-10-0-138-19 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 02:45:20.280488 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.280403 2563 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 02:45:20.285600 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285580 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 02:45:20.285600 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285595 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 02:45:20.285600 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285599 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 02:45:20.285600 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285602 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 02:45:20.285600 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285606 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285609 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285612 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285615 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285618 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285621 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285624 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285626 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285629 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285632 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285634 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285637 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285640 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285642 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285645 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285647 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285650 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285653 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285655 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285658 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 02:45:20.285783 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285661 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285664 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285667 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285669 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285672 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285674 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285677 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285679 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285682 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285684 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285687 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285689 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285692 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285694 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285697 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285700 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285703 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285705 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285708 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 02:45:20.286270 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285711 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285714 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285716 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285719 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285721 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285724 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285727 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285730 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285732 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285735 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285738 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285740 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285743 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285745 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285748 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285750 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285753 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285755 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285758 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 02:45:20.286767 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285762 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285766 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285770 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.285774 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286389 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286394 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286398 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286401 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286404 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286407 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286410 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286413 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286416 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286418 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286421 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286425 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286431 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286434 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286437 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 02:45:20.287251 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286439 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286442 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286444 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286447 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286450 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286816 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286821 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286824 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286827 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286831 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286835 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286838 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286841 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286844 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286846 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286849 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286851 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286854 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286856 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 02:45:20.287700 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286858 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286861 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286864 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286866 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286869 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286871 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286874 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286876 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286879 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286882 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286886 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286888 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286891 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286893 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286896 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286899 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286901 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286903 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286906 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286908 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 02:45:20.288196 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286911 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286914 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286916 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286919 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286921 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286924 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286926 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286930 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286933 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286936 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286938 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286941 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286944 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286946 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286948 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286951 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286953 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286956 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286958 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 02:45:20.288724 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286961 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286964 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286967 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286970 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286973 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286976 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286978 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286981 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286983 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286986 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286988 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286990 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286993 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286995 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.286998 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287000 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287003 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287005 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287008 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287010 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 02:45:20.289202 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287013 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287015 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287018 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287020 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287023 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287025 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287028 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287030 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287032 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287035 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287037 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287040 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.287043 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287875 2563 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287884 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287891 2563 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287895 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287900 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287903 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287907 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287911 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 02:45:20.289759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287914 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287917 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287921 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287924 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287927 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287930 2563 flags.go:64] FLAG: --cgroup-root="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287933 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287936 2563 flags.go:64] FLAG: --client-ca-file="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287939 2563 flags.go:64] FLAG: --cloud-config="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287942 2563 flags.go:64] FLAG: --cloud-provider="external" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287945 2563 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287948 2563 flags.go:64] FLAG: --cluster-domain="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287951 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287954 2563 flags.go:64] FLAG: --config-dir="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287957 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287960 2563 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287965 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287968 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287971 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287974 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287977 2563 flags.go:64] FLAG: --contention-profiling="false" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287980 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287983 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287987 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287990 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 02:45:20.290289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287994 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287997 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.287999 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288002 2563 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288006 2563 flags.go:64] FLAG: --enable-server="true" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288009 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288013 2563 flags.go:64] FLAG: --event-burst="100" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288016 2563 flags.go:64] FLAG: --event-qps="50" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288019 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288022 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288025 2563 flags.go:64] FLAG: --eviction-hard="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288028 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288031 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288034 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288037 2563 flags.go:64] FLAG: --eviction-soft="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288040 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288043 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288046 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288049 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288051 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288054 2563 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288057 2563 flags.go:64] FLAG: --feature-gates="" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288061 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288064 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288067 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 02:45:20.290882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288070 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288073 2563 flags.go:64] FLAG: --healthz-port="10248" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288076 2563 flags.go:64] FLAG: --help="false" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288080 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288083 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288086 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288089 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288093 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288096 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288099 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288102 2563 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288105 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288109 2563 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288112 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288115 2563 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288117 2563 flags.go:64] FLAG: --kube-reserved="" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288120 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288123 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288126 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288129 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288131 2563 flags.go:64] FLAG: --lock-file="" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288134 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288137 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288140 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 02:45:20.291512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288158 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288161 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288163 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288166 2563 flags.go:64] FLAG: --logging-format="text" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288169 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288173 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288176 2563 flags.go:64] FLAG: --manifest-url="" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288179 2563 flags.go:64] FLAG: --manifest-url-header="" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288183 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288190 2563 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288194 2563 flags.go:64] FLAG: --max-pods="110" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288197 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288200 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288203 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288206 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288209 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288212 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288215 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288222 2563 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288225 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288228 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288232 2563 flags.go:64] FLAG: --pod-cidr="" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288235 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 02:45:20.292086 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288241 2563 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288244 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288247 2563 flags.go:64] FLAG: --pods-per-core="0" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288250 2563 flags.go:64] FLAG: --port="10250" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288253 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288256 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05fada01c970438ad" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288259 2563 flags.go:64] FLAG: --qos-reserved="" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288262 2563 flags.go:64] FLAG: --read-only-port="10255" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288265 2563 flags.go:64] FLAG: --register-node="true" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288267 2563 flags.go:64] FLAG: --register-schedulable="true" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288270 2563 flags.go:64] FLAG: --register-with-taints="" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288274 2563 flags.go:64] FLAG: --registry-burst="10" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288277 2563 flags.go:64] FLAG: --registry-qps="5" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288279 2563 flags.go:64] FLAG: --reserved-cpus="" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288282 2563 flags.go:64] FLAG: --reserved-memory="" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288286 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288289 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288292 2563 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288295 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288299 2563 flags.go:64] FLAG: --runonce="false" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288302 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288305 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288308 2563 flags.go:64] FLAG: --seccomp-default="false" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288311 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288313 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288316 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 02:45:20.292688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288320 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288323 2563 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288325 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288328 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288331 2563 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288334 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288337 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288340 2563 flags.go:64] FLAG: --system-cgroups="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288343 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288348 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288351 2563 flags.go:64] FLAG: --tls-cert-file="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288354 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288358 2563 flags.go:64] FLAG: --tls-min-version="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288361 2563 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288363 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288366 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288369 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288372 2563 flags.go:64] FLAG: --v="2" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288376 2563 flags.go:64] FLAG: --version="false" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288380 2563 flags.go:64] FLAG: --vmodule="" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288384 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.288388 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288479 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288483 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 02:45:20.293324 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288486 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288490 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288492 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288495 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288498 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288500 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288502 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288505 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288508 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288511 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288514 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288516 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288519 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288521 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288525 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288527 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288529 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288532 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288535 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288537 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 02:45:20.293892 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288540 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288542 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288544 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288547 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288549 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288552 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288554 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288557 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288559 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288563 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288566 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288569 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288571 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288575 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288577 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288580 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288583 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288585 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288588 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288590 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 02:45:20.294457 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288593 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288596 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288598 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288601 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288603 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288606 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288609 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288612 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288614 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288616 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288620 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288624 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288627 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288629 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288631 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288634 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288637 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288640 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288642 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288645 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 02:45:20.294959 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288647 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288651 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288653 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288656 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288659 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288664 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288668 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288670 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288673 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288676 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288679 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288681 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288684 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288686 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288689 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288691 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288694 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288696 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288699 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 02:45:20.295470 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288702 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288705 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288707 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288710 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.288712 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.289523 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.295476 2563 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.295491 2563 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295534 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295539 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295542 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295545 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295549 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295552 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295554 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295557 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 02:45:20.295936 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295559 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295562 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295564 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295567 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295570 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295572 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295575 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295578 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295580 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295583 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295585 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295588 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295590 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295593 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295595 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295598 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295601 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295603 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295606 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295608 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 02:45:20.296341 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295611 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295613 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295616 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295620 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295623 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295625 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295628 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295630 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295633 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295636 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295638 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295641 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295643 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295645 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295648 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295650 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295653 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295655 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295658 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295660 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 02:45:20.296839 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295663 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295665 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295667 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295670 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295672 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295675 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295678 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295682 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295686 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295689 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295692 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295695 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295698 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295700 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295703 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295706 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295710 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295713 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295715 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 02:45:20.297400 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295718 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295720 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295723 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295725 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295728 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295730 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295733 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295735 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295738 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295740 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295743 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295747 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295750 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295753 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295755 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295758 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295761 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295763 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 02:45:20.297853 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295766 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.295771 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295859 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295863 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295866 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295869 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295872 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295876 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295880 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295882 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295886 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295889 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295893 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295896 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295900 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 02:45:20.298298 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295903 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295905 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295908 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295910 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295913 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295916 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295918 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295920 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295923 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295926 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295928 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295930 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295933 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295935 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295938 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295940 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295943 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295946 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295948 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295951 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 02:45:20.298661 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295954 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295956 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295959 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295962 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295964 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295967 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295969 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295972 2563 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295974 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295977 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295980 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295982 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295985 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295987 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295990 2563 feature_gate.go:328] unrecognized feature gate: Example Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295992 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295995 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.295997 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296000 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296002 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 02:45:20.299157 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296005 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296007 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296009 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296012 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296014 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296017 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296020 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296022 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296025 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296027 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296030 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296032 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296035 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296037 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296040 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296042 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296045 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296047 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296050 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 02:45:20.299637 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296052 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296055 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296057 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296060 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296063 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296065 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296068 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296070 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296072 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296075 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296077 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296080 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296082 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:20.296085 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.296089 2563 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 02:45:20.300083 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.296980 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 02:45:20.301321 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.301307 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 02:45:20.302082 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.302072 2563 server.go:1019] "Starting client certificate rotation" Jun 02 02:45:20.302191 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.302175 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 02:45:20.302232 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.302212 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 02:45:20.321474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.321436 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 02:45:20.326871 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.326858 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 02:45:20.338778 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.338757 2563 log.go:25] "Validated CRI v1 runtime API" Jun 02 02:45:20.343839 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.343825 2563 log.go:25] "Validated CRI v1 image API" Jun 02 02:45:20.346418 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.346404 2563 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 02:45:20.348436 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.348415 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9fbec69f-417f-438d-8918-d51f7dcdf293:/dev/nvme0n1p3 eb7c0246-77a1-4ee2-8f5d-f9cc07826fa0:/dev/nvme0n1p4] Jun 02 02:45:20.348499 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.348434 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 02:45:20.352639 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.352533 2563 manager.go:217] Machine: {Timestamp:2026-06-02 02:45:20.351991091 +0000 UTC m=+0.323591587 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103886 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289e89a5819c2fab5c4e8a0a690096 SystemUUID:ec289e89-a581-9c2f-ab5c-4e8a0a690096 BootID:334d7c5b-8cdf-46a7-89c3-b522ab1a7bf3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:84:c2:4a:82:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:84:c2:4a:82:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:fc:c5:db:a0:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 02:45:20.353231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.353217 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 02:45:20.353309 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.353296 2563 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 02:45:20.354843 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.354825 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 02:45:20.356033 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356012 2563 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 02:45:20.356170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356033 2563 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-19.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 02:45:20.356225 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356180 2563 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 02:45:20.356225 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356189 2563 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 02:45:20.356225 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356196 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 02:45:20.356874 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.356862 2563 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 02:45:20.358219 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.358208 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 02 02:45:20.358465 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.358456 2563 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 02:45:20.360247 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.360238 2563 kubelet.go:491] "Attempting to sync node with API server" Jun 02 02:45:20.360288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.360251 2563 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 02:45:20.360288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.360264 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 02:45:20.360288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.360272 2563 kubelet.go:397] "Adding apiserver pod source" Jun 02 02:45:20.360288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.360280 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 02:45:20.361157 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.361135 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 02:45:20.361199 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.361168 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 02:45:20.363595 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.363574 2563 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 02:45:20.365384 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.365367 2563 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 02:45:20.367391 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367377 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367398 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367405 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367410 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367419 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367426 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367431 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 02:45:20.367439 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367438 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 02:45:20.367616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367454 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 02:45:20.367616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367460 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 02:45:20.367616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367470 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 02:45:20.367616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.367476 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 02:45:20.368157 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.368136 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 02:45:20.368157 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.368158 2563 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 02:45:20.372074 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.372052 2563 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-19.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 02:45:20.372074 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.372055 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-19.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 02:45:20.373259 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.372479 2563 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 02:45:20.373259 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.372533 2563 server.go:1295] "Started kubelet" Jun 02 02:45:20.373259 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.372596 2563 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 02:45:20.373259 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.372984 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 02:45:20.373464 ip-10-0-138-19 systemd[1]: Started Kubernetes Kubelet. Jun 02 02:45:20.373542 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.373425 2563 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 02:45:20.373542 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.373535 2563 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 02:45:20.376684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.376668 2563 server.go:317] "Adding debug handlers to kubelet server" Jun 02 02:45:20.376771 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.376732 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jfckx" Jun 02 02:45:20.376933 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.376912 2563 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 02:45:20.379873 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.379084 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-19.ec2.internal.18b5238df9296ab0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-19.ec2.internal,UID:ip-10-0-138-19.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-19.ec2.internal,},FirstTimestamp:2026-06-02 02:45:20.372501168 +0000 UTC m=+0.344101648,LastTimestamp:2026-06-02 02:45:20.372501168 +0000 UTC m=+0.344101648,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-19.ec2.internal,}" Jun 02 02:45:20.381099 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381085 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 02:45:20.381212 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381109 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 02:45:20.381270 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381249 2563 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 02:45:20.381270 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381259 2563 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 02:45:20.381416 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381337 2563 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 02:45:20.381416 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381398 2563 reconstruct.go:97] "Volume reconstruction finished" Jun 02 02:45:20.381416 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.381406 2563 reconciler.go:26] "Reconciler: start to sync state" Jun 02 02:45:20.381546 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.381516 2563 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 02:45:20.381806 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.381787 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.382037 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382023 2563 factory.go:55] Registering systemd factory Jun 02 02:45:20.382114 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382040 2563 factory.go:223] Registration of the systemd container factory successfully Jun 02 02:45:20.382248 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382234 2563 factory.go:153] Registering CRI-O factory Jun 02 02:45:20.382303 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382252 2563 factory.go:223] Registration of the crio container factory successfully Jun 02 02:45:20.382303 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382297 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 02:45:20.382402 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382310 2563 factory.go:103] Registering Raw factory Jun 02 02:45:20.382402 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.382320 2563 manager.go:1196] Started watching for new ooms in manager Jun 02 02:45:20.383184 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.383166 2563 manager.go:319] Starting recovery of all containers Jun 02 02:45:20.384577 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.384555 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jfckx" Jun 02 02:45:20.391596 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.391455 2563 manager.go:324] Recovery completed Jun 02 02:45:20.391926 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.391911 2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:20.394092 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.394072 2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-19.ec2.internal\" not found" node="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.397313 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.397238 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.399114 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399099 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.399199 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399123 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.399199 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399132 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.399608 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399594 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 02:45:20.399608 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399606 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 02:45:20.399681 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.399621 2563 state_mem.go:36] "Initialized new in-memory state store" Jun 02 02:45:20.401574 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.401562 2563 policy_none.go:49] "None policy: Start" Jun 02 02:45:20.401616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.401578 2563 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 02:45:20.401616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.401587 2563 state_mem.go:35] "Initializing new in-memory state store" Jun 02 02:45:20.436804 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.436792 2563 manager.go:341] "Starting Device Plugin manager" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.436820 2563 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.436829 2563 server.go:85] "Starting device plugin registration server" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.437264 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.437282 2563 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.437396 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.437464 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.437473 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.438095 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 02:45:20.446929 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.438129 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.504642 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.504621 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 02:45:20.505697 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.505683 2563 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 02:45:20.505748 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.505708 2563 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 02:45:20.505748 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.505725 2563 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 02:45:20.505748 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.505735 2563 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 02:45:20.505918 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.505777 2563 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 02:45:20.508968 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.508952 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:20.537682 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.537655 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.538392 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.538377 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.538477 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.538403 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.538477 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.538413 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.538477 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.538434 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.545846 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.545833 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.545901 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.545851 2563 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-19.ec2.internal\": node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.556327 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.556307 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.605914 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.605869 2563 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal"] Jun 02 02:45:20.605980 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.605929 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.607257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.607240 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.607326 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.607268 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.607326 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.607278 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.608573 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.608561 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.608751 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.608736 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.608799 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.608767 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.609215 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609200 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.609293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609201 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.609293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609229 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.609293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609242 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.609293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609251 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.609293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.609263 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.610228 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.610212 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.610318 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.610236 2563 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 02:45:20.610833 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.610817 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientMemory" Jun 02 02:45:20.610911 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.610847 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 02:45:20.610911 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.610859 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeHasSufficientPID" Jun 02 02:45:20.631703 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.631677 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-19.ec2.internal\" not found" node="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.634891 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.634877 2563 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-19.ec2.internal\" not found" node="ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.656501 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.656483 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.757331 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.757310 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.782591 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.782568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.782665 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.782598 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.782665 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.782625 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f223c2f1cad1315f5d568b3ba3c4623-config\") pod \"kube-apiserver-proxy-ip-10-0-138-19.ec2.internal\" (UID: \"9f223c2f1cad1315f5d568b3ba3c4623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.857851 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.857790 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:20.883227 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883206 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.883295 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883232 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f223c2f1cad1315f5d568b3ba3c4623-config\") pod \"kube-apiserver-proxy-ip-10-0-138-19.ec2.internal\" (UID: \"9f223c2f1cad1315f5d568b3ba3c4623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.883295 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883250 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.883362 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883294 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.883362 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883306 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ed121057a006c99d44fec52db04daff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal\" (UID: \"7ed121057a006c99d44fec52db04daff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.883362 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.883317 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f223c2f1cad1315f5d568b3ba3c4623-config\") pod \"kube-apiserver-proxy-ip-10-0-138-19.ec2.internal\" (UID: \"9f223c2f1cad1315f5d568b3ba3c4623\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.934332 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.934309 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.937843 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:20.937826 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:20.958472 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:20.958454 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.058924 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.058890 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.159385 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.159329 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.259798 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.259780 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.302201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.302185 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 02:45:21.302600 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.302319 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 02:45:21.302600 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.302336 2563 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 02:45:21.360500 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.360479 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.381205 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.381187 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 02:45:21.386537 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.386496 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 02:40:20 +0000 UTC" deadline="2027-12-30 13:35:05.463794931 +0000 UTC" Jun 02 02:45:21.386608 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.386539 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13834h49m44.07726231s" Jun 02 02:45:21.390929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.390909 2563 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 02:45:21.408681 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:21.408651 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f223c2f1cad1315f5d568b3ba3c4623.slice/crio-aef166e6b609843d554d88f11ffbb843e9843eb34a67194ad6901cf5b9f2d0ed WatchSource:0}: Error finding container aef166e6b609843d554d88f11ffbb843e9843eb34a67194ad6901cf5b9f2d0ed: Status 404 returned error can't find the container with id aef166e6b609843d554d88f11ffbb843e9843eb34a67194ad6901cf5b9f2d0ed Jun 02 02:45:21.408843 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:21.408821 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed121057a006c99d44fec52db04daff.slice/crio-9393567fe64dbc3657b875ae8998c5b78d1b0487adeaf07a38db25229b572a9c WatchSource:0}: Error finding container 9393567fe64dbc3657b875ae8998c5b78d1b0487adeaf07a38db25229b572a9c: Status 404 returned error can't find the container with id 9393567fe64dbc3657b875ae8998c5b78d1b0487adeaf07a38db25229b572a9c Jun 02 02:45:21.411931 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.411913 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 02:45:21.412312 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.412283 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gs4lk" Jun 02 02:45:21.419548 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.419530 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gs4lk" Jun 02 02:45:21.446781 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.445674 2563 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:21.460591 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.460570 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.508619 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.508574 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" event={"ID":"7ed121057a006c99d44fec52db04daff","Type":"ContainerStarted","Data":"9393567fe64dbc3657b875ae8998c5b78d1b0487adeaf07a38db25229b572a9c"} Jun 02 02:45:21.509421 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.509404 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" event={"ID":"9f223c2f1cad1315f5d568b3ba3c4623","Type":"ContainerStarted","Data":"aef166e6b609843d554d88f11ffbb843e9843eb34a67194ad6901cf5b9f2d0ed"} Jun 02 02:45:21.561600 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.561580 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.662052 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.662003 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.762460 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.762438 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.863048 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.863016 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.963588 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:21.963520 2563 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-19.ec2.internal\" not found" Jun 02 02:45:21.974486 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.974463 2563 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:21.981491 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.981466 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" Jun 02 02:45:21.993411 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.993390 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 02:45:21.994371 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:21.994354 2563 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" Jun 02 02:45:22.003118 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.003099 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 02:45:22.153120 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.153094 2563 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:22.361584 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.361511 2563 apiserver.go:52] "Watching apiserver" Jun 02 02:45:22.368255 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.368197 2563 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 02:45:22.370016 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.369988 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2zfx6","openshift-cluster-node-tuning-operator/tuned-7wmcw","openshift-dns/node-resolver-hwplc","openshift-image-registry/node-ca-gd2qn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal","openshift-multus/multus-additional-cni-plugins-fhbxx","openshift-network-operator/iptables-alerter-xqk88","kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9","openshift-multus/multus-twg7x","openshift-multus/network-metrics-daemon-rml5c","openshift-network-diagnostics/network-check-target-vzm8k","openshift-ovn-kubernetes/ovnkube-node-hd4s2"] Jun 02 02:45:22.372832 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.372799 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.373926 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.373900 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.374992 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.374973 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.375095 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.375072 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.375684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.375663 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.375684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.375677 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 02:45:22.375829 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.375664 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.375829 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.375670 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pvzbp\"" Jun 02 02:45:22.376515 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.376496 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.376625 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.376553 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.376625 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.376560 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wkpx2\"" Jun 02 02:45:22.377268 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377226 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.377353 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377328 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.377873 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377712 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f4q6l\"" Jun 02 02:45:22.377873 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377738 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 02:45:22.377873 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377744 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.377873 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377839 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.378122 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.377956 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.378658 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.378481 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-c6d95\"" Jun 02 02:45:22.378744 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.378695 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.379442 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379425 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.379525 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379510 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.379609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379589 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 02:45:22.379609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379605 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 02:45:22.379910 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379892 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m86qz\"" Jun 02 02:45:22.379910 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.379904 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 02:45:22.380398 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.380284 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.381264 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.381014 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 02:45:22.381671 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.381453 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 02:45:22.382224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.382197 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kctnl\"" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384158 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384329 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384358 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384331 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.384442 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384578 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pqbcw\"" Jun 02 02:45:22.384737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.384585 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.385937 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.385912 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:22.386027 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.386003 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:22.386994 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.386905 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 02:45:22.386994 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.386911 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m2wb6\"" Jun 02 02:45:22.387386 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.387369 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.389401 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389383 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 02:45:22.390023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389935 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 02:45:22.390023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389960 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbw5t\"" Jun 02 02:45:22.390023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389965 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 02:45:22.390023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389998 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 02:45:22.390291 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.389998 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 02:45:22.390291 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.390066 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 02:45:22.391337 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-registration-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.391414 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-k8s-cni-cncf-io\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391414 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391360 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-multus\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391414 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blxqq\" (UniqueName: \"kubernetes.io/projected/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-kube-api-access-blxqq\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391414 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391397 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b79b5d7-0ab0-4743-9c8a-c6593120483c-host-slash\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.391607 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb382c8c-5050-49a7-a71f-75325166bbc9-host\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.391607 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391514 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztld9\" (UniqueName: \"kubernetes.io/projected/cb382c8c-5050-49a7-a71f-75325166bbc9-kube-api-access-ztld9\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.391607 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-system-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391607 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cnibin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391620 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-os-release\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysconfig\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-kubernetes\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-host\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-conf-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb382c8c-5050-49a7-a71f-75325166bbc9-serviceca\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.391805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-kubelet\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391902 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.391975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-conf\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-lib-modules\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392038 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-tmp\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-hosts-file\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-tmp-dir\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f16ec42c-bc37-4478-8319-ea326c7f5ed6-konnectivity-ca\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-hostroot\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-var-lib-kubelet\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kq6b\" (UniqueName: \"kubernetes.io/projected/2b79b5d7-0ab0-4743-9c8a-c6593120483c-kube-api-access-8kq6b\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392293 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392315 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-sys\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392337 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbp2m\" (UniqueName: \"kubernetes.io/projected/0110ac79-5ae3-4da0-9700-60cd268dc3a9-kube-api-access-kbp2m\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-socket-dir-parent\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-bin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392438 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-modprobe-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-tuned\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392504 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b79b5d7-0ab0-4743-9c8a-c6593120483c-iptables-alerter-script\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392534 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392611 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-netns\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-multus-certs\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.392746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-systemd\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-os-release\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392762 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r6n7\" (UniqueName: \"kubernetes.io/projected/ad482e70-618c-45fa-83d5-d7ad477d7d50-kube-api-access-9r6n7\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392787 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-etc-kubernetes\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-run\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cnibin\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392861 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f16ec42c-bc37-4478-8319-ea326c7f5ed6-agent-certs\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392889 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cni-binary-copy\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-device-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk5gf\" (UniqueName: \"kubernetes.io/projected/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-kube-api-access-vk5gf\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-socket-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.392999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-sys-fs\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.393019 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-daemon-config\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.393534 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.393033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nblzb\" (UniqueName: \"kubernetes.io/projected/c91fbee6-c26d-4066-a2dd-a911690ef5e4-kube-api-access-nblzb\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.420891 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.420867 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 02:40:21 +0000 UTC" deadline="2027-11-14 11:56:46.415967443 +0000 UTC" Jun 02 02:45:22.420891 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.420888 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12729h11m23.99508115s" Jun 02 02:45:22.482795 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.482773 2563 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 02:45:22.493343 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493320 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-modprobe-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-systemd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.493474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-var-lib-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.493474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493441 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493472 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-tuned\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493489 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b79b5d7-0ab0-4743-9c8a-c6593120483c-iptables-alerter-script\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493501 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-modprobe-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493513 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493548 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-netns\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493576 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-multus-certs\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493584 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-d\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493571 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493603 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-systemd\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493635 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-os-release\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493643 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-netns\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-systemd\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493660 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-multus-certs\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.493692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493686 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9r6n7\" (UniqueName: \"kubernetes.io/projected/ad482e70-618c-45fa-83d5-d7ad477d7d50-kube-api-access-9r6n7\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493710 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-etc-kubernetes\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493717 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-os-release\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493733 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-run\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493755 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cnibin\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493779 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f16ec42c-bc37-4478-8319-ea326c7f5ed6-agent-certs\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493810 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-kubelet\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493835 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493862 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493992 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cnibin\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.493995 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-etc-kubernetes\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494008 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494034 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cni-binary-copy\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494040 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-run\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494063 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-device-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494090 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vk5gf\" (UniqueName: \"kubernetes.io/projected/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-kube-api-access-vk5gf\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494115 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-socket-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.494369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-device-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494171 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b79b5d7-0ab0-4743-9c8a-c6593120483c-iptables-alerter-script\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494204 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-sys-fs\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-etc-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdphs\" (UniqueName: \"kubernetes.io/projected/513f443e-a2f6-401d-a082-4222d02fdcd0-kube-api-access-mdphs\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494305 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-socket-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494316 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-daemon-config\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494340 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nblzb\" (UniqueName: \"kubernetes.io/projected/c91fbee6-c26d-4066-a2dd-a911690ef5e4-kube-api-access-nblzb\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-slash\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494490 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-sys-fs\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494528 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-registration-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494565 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-k8s-cni-cncf-io\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494601 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-multus\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494626 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-blxqq\" (UniqueName: \"kubernetes.io/projected/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-kube-api-access-blxqq\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494651 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b79b5d7-0ab0-4743-9c8a-c6593120483c-host-slash\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494660 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-run-k8s-cni-cncf-io\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495224 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494679 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb382c8c-5050-49a7-a71f-75325166bbc9-host\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494689 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cni-binary-copy\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494719 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb382c8c-5050-49a7-a71f-75325166bbc9-host\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494738 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-multus\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494758 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztld9\" (UniqueName: \"kubernetes.io/projected/cb382c8c-5050-49a7-a71f-75325166bbc9-kube-api-access-ztld9\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494780 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-registration-dir\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494812 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-system-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494827 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cnibin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494854 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-os-release\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494878 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysconfig\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494887 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-cnibin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494887 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b79b5d7-0ab0-4743-9c8a-c6593120483c-host-slash\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494891 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494907 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-kubernetes\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-systemd-units\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494944 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-daemon-config\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494949 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-system-cni-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.495953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-netd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.494994 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-kubernetes\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495003 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysconfig\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495033 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-os-release\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495079 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-host\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495126 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-config\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-env-overrides\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-host\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495253 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-conf-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495289 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb382c8c-5050-49a7-a71f-75325166bbc9-serviceca\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495311 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-conf-dir\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495350 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s68m5\" (UniqueName: \"kubernetes.io/projected/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-kube-api-access-s68m5\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495375 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-netns\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495401 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495438 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-kubelet\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495466 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495490 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-conf\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.496802 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495514 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-lib-modules\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495572 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-tmp\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495631 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-hosts-file\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495659 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-tmp-dir\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495686 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f16ec42c-bc37-4478-8319-ea326c7f5ed6-konnectivity-ca\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495724 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cb382c8c-5050-49a7-a71f-75325166bbc9-serviceca\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495715 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495764 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-hostroot\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495793 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-ovn\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495817 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-log-socket\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495848 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-sysctl-conf\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495916 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-var-lib-kubelet\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.495938 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-lib-modules\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496052 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496092 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-kubelet\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496194 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-hostroot\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.497641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496259 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-hosts-file\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496465 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-node-log\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-bin\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496520 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-tmp-dir\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496586 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-script-lib\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496616 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-var-lib-kubelet\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496663 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8kq6b\" (UniqueName: \"kubernetes.io/projected/2b79b5d7-0ab0-4743-9c8a-c6593120483c-kube-api-access-8kq6b\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496693 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496719 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-sys\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496748 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496798 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kbp2m\" (UniqueName: \"kubernetes.io/projected/0110ac79-5ae3-4da0-9700-60cd268dc3a9-kube-api-access-kbp2m\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/513f443e-a2f6-401d-a082-4222d02fdcd0-ovn-node-metrics-cert\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496857 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-socket-dir-parent\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496884 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-bin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.496947 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-host-var-lib-cni-bin\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497019 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad482e70-618c-45fa-83d5-d7ad477d7d50-etc-selinux\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.498359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497083 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c91fbee6-c26d-4066-a2dd-a911690ef5e4-sys\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497131 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-multus-socket-dir-parent\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497191 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0110ac79-5ae3-4da0-9700-60cd268dc3a9-system-cni-dir\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497391 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0110ac79-5ae3-4da0-9700-60cd268dc3a9-cni-binary-copy\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-etc-tuned\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.499194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.497827 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f16ec42c-bc37-4478-8319-ea326c7f5ed6-agent-certs\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.499420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.499385 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c91fbee6-c26d-4066-a2dd-a911690ef5e4-tmp\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.504069 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.503977 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztld9\" (UniqueName: \"kubernetes.io/projected/cb382c8c-5050-49a7-a71f-75325166bbc9-kube-api-access-ztld9\") pod \"node-ca-gd2qn\" (UID: \"cb382c8c-5050-49a7-a71f-75325166bbc9\") " pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.505692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.504129 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk5gf\" (UniqueName: \"kubernetes.io/projected/bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c-kube-api-access-vk5gf\") pod \"node-resolver-hwplc\" (UID: \"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c\") " pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.505692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.505116 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nblzb\" (UniqueName: \"kubernetes.io/projected/c91fbee6-c26d-4066-a2dd-a911690ef5e4-kube-api-access-nblzb\") pod \"tuned-7wmcw\" (UID: \"c91fbee6-c26d-4066-a2dd-a911690ef5e4\") " pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.505692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.505552 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blxqq\" (UniqueName: \"kubernetes.io/projected/4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f-kube-api-access-blxqq\") pod \"multus-twg7x\" (UID: \"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f\") " pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.506198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.506129 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kq6b\" (UniqueName: \"kubernetes.io/projected/2b79b5d7-0ab0-4743-9c8a-c6593120483c-kube-api-access-8kq6b\") pod \"iptables-alerter-xqk88\" (UID: \"2b79b5d7-0ab0-4743-9c8a-c6593120483c\") " pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.506393 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.506370 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f16ec42c-bc37-4478-8319-ea326c7f5ed6-konnectivity-ca\") pod \"konnectivity-agent-2zfx6\" (UID: \"f16ec42c-bc37-4478-8319-ea326c7f5ed6\") " pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.506896 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.506837 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbp2m\" (UniqueName: \"kubernetes.io/projected/0110ac79-5ae3-4da0-9700-60cd268dc3a9-kube-api-access-kbp2m\") pod \"multus-additional-cni-plugins-fhbxx\" (UID: \"0110ac79-5ae3-4da0-9700-60cd268dc3a9\") " pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.507457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.507426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r6n7\" (UniqueName: \"kubernetes.io/projected/ad482e70-618c-45fa-83d5-d7ad477d7d50-kube-api-access-9r6n7\") pod \"aws-ebs-csi-driver-node-ff7c9\" (UID: \"ad482e70-618c-45fa-83d5-d7ad477d7d50\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.597453 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597412 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-kubelet\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597453 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597450 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597477 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-etc-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597503 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mdphs\" (UniqueName: \"kubernetes.io/projected/513f443e-a2f6-401d-a082-4222d02fdcd0-kube-api-access-mdphs\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597525 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597538 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-kubelet\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597548 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-slash\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597595 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-slash\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597623 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-systemd-units\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597650 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.597654 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:22.597677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597675 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-netd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-systemd-units\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597699 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-config\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.597725 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:23.09770602 +0000 UTC m=+3.069306487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597652 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-etc-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597758 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-env-overrides\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597789 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s68m5\" (UniqueName: \"kubernetes.io/projected/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-kube-api-access-s68m5\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597726 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597779 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-netd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597819 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-netns\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597853 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597896 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-ovn\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597915 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-log-socket\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597915 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-run-netns\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597937 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597967 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-node-log\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598186 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597991 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-bin\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598003 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-log-socket\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.597966 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-ovn\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598015 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598022 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-script-lib\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598051 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/513f443e-a2f6-401d-a082-4222d02fdcd0-ovn-node-metrics-cert\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598063 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-host-cni-bin\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598079 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-systemd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598102 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-var-lib-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598054 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-node-log\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598191 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-var-lib-openvswitch\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598232 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/513f443e-a2f6-401d-a082-4222d02fdcd0-run-systemd\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598358 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-config\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598367 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-env-overrides\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.598990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.598567 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/513f443e-a2f6-401d-a082-4222d02fdcd0-ovnkube-script-lib\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.600456 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.600426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/513f443e-a2f6-401d-a082-4222d02fdcd0-ovn-node-metrics-cert\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.606809 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.606789 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:22.606809 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.606813 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:22.606958 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.606826 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:22.606958 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:22.606943 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:23.106920858 +0000 UTC m=+3.078521334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:22.609340 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.609322 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s68m5\" (UniqueName: \"kubernetes.io/projected/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-kube-api-access-s68m5\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:22.609543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.609528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdphs\" (UniqueName: \"kubernetes.io/projected/513f443e-a2f6-401d-a082-4222d02fdcd0-kube-api-access-mdphs\") pod \"ovnkube-node-hd4s2\" (UID: \"513f443e-a2f6-401d-a082-4222d02fdcd0\") " pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.686669 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.686587 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xqk88" Jun 02 02:45:22.694248 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.694226 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" Jun 02 02:45:22.701941 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.701925 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hwplc" Jun 02 02:45:22.706482 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.706468 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gd2qn" Jun 02 02:45:22.712966 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.712949 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" Jun 02 02:45:22.719462 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.719443 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:22.725923 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.725902 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" Jun 02 02:45:22.732444 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.732429 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-twg7x" Jun 02 02:45:22.738085 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.738069 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:22.853551 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:22.853523 2563 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 02:45:22.963177 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.963131 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0110ac79_5ae3_4da0_9700_60cd268dc3a9.slice/crio-cf73aa8cc2474182ae5bfc28a91ad77d2fefd31963d60ff0313f08b8379555ee WatchSource:0}: Error finding container cf73aa8cc2474182ae5bfc28a91ad77d2fefd31963d60ff0313f08b8379555ee: Status 404 returned error can't find the container with id cf73aa8cc2474182ae5bfc28a91ad77d2fefd31963d60ff0313f08b8379555ee Jun 02 02:45:22.965008 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.964881 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa5c10b_7e5f_4a4b_9fbd_824fd3215f6f.slice/crio-79a25db3802dc963f65641da255d4b37b75f92b1da2a63995ca775eda10f3747 WatchSource:0}: Error finding container 79a25db3802dc963f65641da255d4b37b75f92b1da2a63995ca775eda10f3747: Status 404 returned error can't find the container with id 79a25db3802dc963f65641da255d4b37b75f92b1da2a63995ca775eda10f3747 Jun 02 02:45:22.967696 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.967676 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb382c8c_5050_49a7_a71f_75325166bbc9.slice/crio-15b135a6e1f39696fb2e8c89a23ec70950e2fe8ce9c8137b6feea6a24fcc36eb WatchSource:0}: Error finding container 15b135a6e1f39696fb2e8c89a23ec70950e2fe8ce9c8137b6feea6a24fcc36eb: Status 404 returned error can't find the container with id 15b135a6e1f39696fb2e8c89a23ec70950e2fe8ce9c8137b6feea6a24fcc36eb Jun 02 02:45:22.970312 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.968711 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b79b5d7_0ab0_4743_9c8a_c6593120483c.slice/crio-337af1d91b807277efa43d9993b70310b773de9713d6a9c4021a464f27c47b49 WatchSource:0}: Error finding container 337af1d91b807277efa43d9993b70310b773de9713d6a9c4021a464f27c47b49: Status 404 returned error can't find the container with id 337af1d91b807277efa43d9993b70310b773de9713d6a9c4021a464f27c47b49 Jun 02 02:45:22.970312 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.969996 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16ec42c_bc37_4478_8319_ea326c7f5ed6.slice/crio-65479ca4b209005e2db65c0f042f7429424761957189132e495a82a06b742228 WatchSource:0}: Error finding container 65479ca4b209005e2db65c0f042f7429424761957189132e495a82a06b742228: Status 404 returned error can't find the container with id 65479ca4b209005e2db65c0f042f7429424761957189132e495a82a06b742228 Jun 02 02:45:22.971629 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.971601 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc76ee11_6d2c_4cf2_ac0f_b8f35ba0865c.slice/crio-41695159851d9799c620b64a52d60e1674c681cfe62cb35f63e8b16a6be10016 WatchSource:0}: Error finding container 41695159851d9799c620b64a52d60e1674c681cfe62cb35f63e8b16a6be10016: Status 404 returned error can't find the container with id 41695159851d9799c620b64a52d60e1674c681cfe62cb35f63e8b16a6be10016 Jun 02 02:45:22.973436 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.973413 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad482e70_618c_45fa_83d5_d7ad477d7d50.slice/crio-4c5c0d24113414ca1931bb932d2c12605d37fd94a9ed17055c5a738c9ef4daf2 WatchSource:0}: Error finding container 4c5c0d24113414ca1931bb932d2c12605d37fd94a9ed17055c5a738c9ef4daf2: Status 404 returned error can't find the container with id 4c5c0d24113414ca1931bb932d2c12605d37fd94a9ed17055c5a738c9ef4daf2 Jun 02 02:45:22.974735 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.974395 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513f443e_a2f6_401d_a082_4222d02fdcd0.slice/crio-162e5e9ccd1183b45369229104ebbf7babb42a88744799bd1f303bdb7685a97b WatchSource:0}: Error finding container 162e5e9ccd1183b45369229104ebbf7babb42a88744799bd1f303bdb7685a97b: Status 404 returned error can't find the container with id 162e5e9ccd1183b45369229104ebbf7babb42a88744799bd1f303bdb7685a97b Jun 02 02:45:22.975499 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:22.975477 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91fbee6_c26d_4066_a2dd_a911690ef5e4.slice/crio-9b69173135185882cbca41307299359ea1abd20985181cbd4611257673e62ba1 WatchSource:0}: Error finding container 9b69173135185882cbca41307299359ea1abd20985181cbd4611257673e62ba1: Status 404 returned error can't find the container with id 9b69173135185882cbca41307299359ea1abd20985181cbd4611257673e62ba1 Jun 02 02:45:23.102824 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.102693 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:23.102928 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.102835 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:23.102928 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.102889 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:24.102871696 +0000 UTC m=+4.074472165 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:23.203070 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.202990 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:23.203193 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.203137 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:23.203193 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.203176 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:23.203193 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.203186 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:23.203288 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.203233 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:24.203219804 +0000 UTC m=+4.174820266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:23.421530 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.421446 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 02:40:21 +0000 UTC" deadline="2028-01-02 17:19:20.550672189 +0000 UTC" Jun 02 02:45:23.421530 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.421485 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13910h33m57.129191486s" Jun 02 02:45:23.506538 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.505972 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:23.506538 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:23.506082 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:23.524559 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.524527 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" event={"ID":"9f223c2f1cad1315f5d568b3ba3c4623","Type":"ContainerStarted","Data":"552e3c60ced7805ee7929c386ddd54cbc4017160d0b2f37a8d080f1608ac9437"} Jun 02 02:45:23.532871 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.532844 2563 generic.go:358] "Generic (PLEG): container finished" podID="7ed121057a006c99d44fec52db04daff" containerID="4cb4c8513838129af15195d0bd6b00f9a2227791bd2dea5a45cf6d44334c134c" exitCode=0 Jun 02 02:45:23.533113 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.533076 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" event={"ID":"7ed121057a006c99d44fec52db04daff","Type":"ContainerDied","Data":"4cb4c8513838129af15195d0bd6b00f9a2227791bd2dea5a45cf6d44334c134c"} Jun 02 02:45:23.539157 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.539099 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-19.ec2.internal" podStartSLOduration=2.539084695 podStartE2EDuration="2.539084695s" podCreationTimestamp="2026-06-02 02:45:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:45:23.538309912 +0000 UTC m=+3.509910397" watchObservedRunningTime="2026-06-02 02:45:23.539084695 +0000 UTC m=+3.510685184" Jun 02 02:45:23.547493 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.546741 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" event={"ID":"ad482e70-618c-45fa-83d5-d7ad477d7d50","Type":"ContainerStarted","Data":"4c5c0d24113414ca1931bb932d2c12605d37fd94a9ed17055c5a738c9ef4daf2"} Jun 02 02:45:23.555557 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.555509 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2zfx6" event={"ID":"f16ec42c-bc37-4478-8319-ea326c7f5ed6","Type":"ContainerStarted","Data":"65479ca4b209005e2db65c0f042f7429424761957189132e495a82a06b742228"} Jun 02 02:45:23.558978 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.558953 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gd2qn" event={"ID":"cb382c8c-5050-49a7-a71f-75325166bbc9","Type":"ContainerStarted","Data":"15b135a6e1f39696fb2e8c89a23ec70950e2fe8ce9c8137b6feea6a24fcc36eb"} Jun 02 02:45:23.581540 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.581515 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerStarted","Data":"cf73aa8cc2474182ae5bfc28a91ad77d2fefd31963d60ff0313f08b8379555ee"} Jun 02 02:45:23.587318 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.587076 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" event={"ID":"c91fbee6-c26d-4066-a2dd-a911690ef5e4","Type":"ContainerStarted","Data":"9b69173135185882cbca41307299359ea1abd20985181cbd4611257673e62ba1"} Jun 02 02:45:23.591689 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.591665 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"162e5e9ccd1183b45369229104ebbf7babb42a88744799bd1f303bdb7685a97b"} Jun 02 02:45:23.600864 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.600815 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hwplc" event={"ID":"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c","Type":"ContainerStarted","Data":"41695159851d9799c620b64a52d60e1674c681cfe62cb35f63e8b16a6be10016"} Jun 02 02:45:23.627843 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.627795 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqk88" event={"ID":"2b79b5d7-0ab0-4743-9c8a-c6593120483c","Type":"ContainerStarted","Data":"337af1d91b807277efa43d9993b70310b773de9713d6a9c4021a464f27c47b49"} Jun 02 02:45:23.653795 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:23.653747 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-twg7x" event={"ID":"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f","Type":"ContainerStarted","Data":"79a25db3802dc963f65641da255d4b37b75f92b1da2a63995ca775eda10f3747"} Jun 02 02:45:24.110871 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.110791 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:24.111047 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.111001 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:24.111132 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.111069 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:26.111051183 +0000 UTC m=+6.082651654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:24.212210 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.211822 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:24.212210 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.212008 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:24.212210 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.212027 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:24.212210 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.212042 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:24.212210 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.212101 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:26.212084033 +0000 UTC m=+6.183684510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:24.508466 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.508435 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:24.508934 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.508629 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:24.645349 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.644525 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qrl6f"] Jun 02 02:45:24.646884 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.646457 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.646884 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.646533 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:24.666532 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.666459 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" event={"ID":"7ed121057a006c99d44fec52db04daff","Type":"ContainerStarted","Data":"6fbc9632534f27141a9f60cbd7efc480221cf9ed38efe8c5fe68a28f4d5304fd"} Jun 02 02:45:24.715945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.715770 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-dbus\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.715945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.715844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-kubelet-config\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.715945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.715874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.816894 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-dbus\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.816972 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-kubelet-config\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.817001 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.817111 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-dbus\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.817129 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:24.817208 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-kubelet-config\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:24.817296 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:24.817213 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:25.317191764 +0000 UTC m=+5.288792242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:25.322811 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:25.322774 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:25.323022 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:25.322937 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:25.323022 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:25.323013 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:26.322995207 +0000 UTC m=+6.294595687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:25.506399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:25.506368 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:25.506555 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:25.506488 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:26.129606 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:26.129558 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:26.130069 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.129736 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:26.130069 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.129813 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:30.129793514 +0000 UTC m=+10.101393978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:26.230615 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:26.230550 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:26.230789 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.230710 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:26.230789 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.230735 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:26.230789 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.230749 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:26.230948 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.230813 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:30.230792833 +0000 UTC m=+10.202393301 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:26.331879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:26.331843 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:26.332047 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.332029 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:26.332108 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.332098 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:28.332079302 +0000 UTC m=+8.303679772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:26.506458 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:26.506378 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:26.506611 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.506518 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:26.506874 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:26.506853 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:26.506989 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:26.506968 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:27.506365 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:27.506046 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:27.506365 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:27.506192 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:28.355772 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:28.355730 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:28.355960 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:28.355885 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:28.355960 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:28.355955 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:32.355935954 +0000 UTC m=+12.327536434 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:28.506865 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:28.506717 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:28.506865 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:28.506849 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:28.507345 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:28.507328 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:28.507446 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:28.507428 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:29.506927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:29.506894 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:29.507408 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:29.507029 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:30.168133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:30.168095 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:30.168335 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.168255 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:30.168335 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.168321 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:38.16830096 +0000 UTC m=+18.139901466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:30.269347 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:30.269286 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:30.269515 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.269471 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:30.269515 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.269496 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:30.269515 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.269509 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:30.269686 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.269566 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:38.269547621 +0000 UTC m=+18.241148086 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:30.509368 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:30.509334 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:30.509674 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:30.509355 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:30.509674 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.509480 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:30.509674 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:30.509545 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:31.506695 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:31.506658 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:31.506861 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:31.506767 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:32.383772 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:32.383729 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:32.384273 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:32.383887 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:32.384273 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:32.383959 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:40.383939757 +0000 UTC m=+20.355540224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:32.506846 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:32.506814 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:32.507003 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:32.506944 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:32.507074 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:32.507003 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:32.507130 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:32.507115 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:33.506847 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:33.506808 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:33.507266 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:33.506930 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:34.505943 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:34.505913 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:34.506099 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:34.505958 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:34.506099 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:34.506028 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:34.506241 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:34.506163 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:35.506931 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:35.506898 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:35.507349 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:35.507003 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:36.506395 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:36.506332 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:36.506395 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:36.506377 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:36.506617 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:36.506474 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:36.506617 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:36.506602 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:37.506950 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:37.506892 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:37.507390 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:37.507044 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:38.227974 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:38.227944 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:38.228160 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.228085 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:38.228209 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.228171 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:54.228135746 +0000 UTC m=+34.199736214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:38.328428 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:38.328395 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:38.328605 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.328537 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:38.328605 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.328553 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:38.328605 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.328562 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:38.328746 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.328618 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:54.328599471 +0000 UTC m=+34.300199948 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:38.506181 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:38.506091 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:38.506323 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.506244 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:38.506323 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:38.506298 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:38.506442 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:38.506416 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:39.506070 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:39.506034 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:39.506616 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:39.506200 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:40.445074 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:40.445043 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:40.445243 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:40.445169 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:40.445243 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:40.445212 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret podName:d8a3dee4-d352-4d4a-a453-62cea3c0ce58 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:56.445199978 +0000 UTC m=+36.416800442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret") pod "global-pull-secret-syncer-qrl6f" (UID: "d8a3dee4-d352-4d4a-a453-62cea3c0ce58") : object "kube-system"/"original-pull-secret" not registered Jun 02 02:45:40.507021 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:40.506993 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:40.507379 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:40.507081 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:40.507379 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:40.507185 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:40.507379 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:40.507293 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:41.506543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.506341 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:41.506630 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:41.506614 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:41.709943 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.709907 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-twg7x" event={"ID":"4aa5c10b-7e5f-4a4b-9fbd-824fd3215f6f","Type":"ContainerStarted","Data":"23a19cda9c92a41324a3758c51093489f30f43962823c29f8f58a8e6899f154e"} Jun 02 02:45:41.711622 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.711596 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" event={"ID":"ad482e70-618c-45fa-83d5-d7ad477d7d50","Type":"ContainerStarted","Data":"f73baafe1d938c65948d096574012ada63ea76bb69d7f85fd25da498e202e51f"} Jun 02 02:45:41.713034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.713006 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2zfx6" event={"ID":"f16ec42c-bc37-4478-8319-ea326c7f5ed6","Type":"ContainerStarted","Data":"67017af6c4c027e02953ffd8b39d4bb2ca3a6b8ba0d890876ccb77780763b988"} Jun 02 02:45:41.715851 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.715824 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gd2qn" event={"ID":"cb382c8c-5050-49a7-a71f-75325166bbc9","Type":"ContainerStarted","Data":"f5eb9dac07e8fb0397d4c35f133d415c51acb61ddba41a682063d7eb8d657268"} Jun 02 02:45:41.717446 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.717416 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="3b14fd851f9e1a7557f1d0aea645c217dc737f850ed5357559b13b2635002ae1" exitCode=0 Jun 02 02:45:41.717533 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.717485 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"3b14fd851f9e1a7557f1d0aea645c217dc737f850ed5357559b13b2635002ae1"} Jun 02 02:45:41.719080 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.719052 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" event={"ID":"c91fbee6-c26d-4066-a2dd-a911690ef5e4","Type":"ContainerStarted","Data":"a6c5cf8cc2b5b7205f0c9a9d03f284386a69722cddc6df4d0edec88a1a11c25e"} Jun 02 02:45:41.721752 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.721736 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:45:41.722051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722025 2563 generic.go:358] "Generic (PLEG): container finished" podID="513f443e-a2f6-401d-a082-4222d02fdcd0" containerID="26163de22c036f3786d3e32dc8e008935e31337b466fede7479f026b8c1bb778" exitCode=1 Jun 02 02:45:41.722164 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722084 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"386ebb51ff2e078288ae506fe37c5ca13abddbaa53674df42421d48635385ffb"} Jun 02 02:45:41.722164 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722113 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"9b45d82dc9db2d2791fdabfbf402c47d6ad9e1343fd126189e755c67cbc6e663"} Jun 02 02:45:41.722164 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722127 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"ed9f9a0abcd119beeb04f2b113ebfc7d6f31f5b0c1cfb831f625051e91106ca2"} Jun 02 02:45:41.722164 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722153 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"9ac3a14f7839a7b9f5bd3157b929e68f795e66d1631bb6c2b31b5e94590f1036"} Jun 02 02:45:41.722295 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722166 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerDied","Data":"26163de22c036f3786d3e32dc8e008935e31337b466fede7479f026b8c1bb778"} Jun 02 02:45:41.722295 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.722181 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"76137a43e88d7c76d2cb947b47506662d2e18660099db8361ce8b91024bfc2c3"} Jun 02 02:45:41.723394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.723371 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hwplc" event={"ID":"bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c","Type":"ContainerStarted","Data":"bbff6867a4ebc2b62dda9860d6ca2d5eb2f753bab0cdcaeb7425532b73c9eab7"} Jun 02 02:45:41.727692 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.727648 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-twg7x" podStartSLOduration=3.778890324 podStartE2EDuration="21.727618032s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.966883908 +0000 UTC m=+2.938484371" lastFinishedPulling="2026-06-02 02:45:40.915611601 +0000 UTC m=+20.887212079" observedRunningTime="2026-06-02 02:45:41.727521379 +0000 UTC m=+21.699121876" watchObservedRunningTime="2026-06-02 02:45:41.727618032 +0000 UTC m=+21.699218509" Jun 02 02:45:41.728407 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.728373 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-19.ec2.internal" podStartSLOduration=20.728363914 podStartE2EDuration="20.728363914s" podCreationTimestamp="2026-06-02 02:45:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:45:24.686992902 +0000 UTC m=+4.658593388" watchObservedRunningTime="2026-06-02 02:45:41.728363914 +0000 UTC m=+21.699964398" Jun 02 02:45:41.745556 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.745520 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7wmcw" podStartSLOduration=3.82758271 podStartE2EDuration="21.745511145s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.977490074 +0000 UTC m=+2.949090540" lastFinishedPulling="2026-06-02 02:45:40.895418506 +0000 UTC m=+20.867018975" observedRunningTime="2026-06-02 02:45:41.744945429 +0000 UTC m=+21.716545938" watchObservedRunningTime="2026-06-02 02:45:41.745511145 +0000 UTC m=+21.717111630" Jun 02 02:45:41.758986 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.758955 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-hwplc" podStartSLOduration=3.840222427 podStartE2EDuration="21.758945419s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.974112783 +0000 UTC m=+2.945713254" lastFinishedPulling="2026-06-02 02:45:40.892835766 +0000 UTC m=+20.864436246" observedRunningTime="2026-06-02 02:45:41.758756143 +0000 UTC m=+21.730356631" watchObservedRunningTime="2026-06-02 02:45:41.758945419 +0000 UTC m=+21.730545904" Jun 02 02:45:41.803028 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.802988 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gd2qn" podStartSLOduration=3.879858961 podStartE2EDuration="21.802978361s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.96983898 +0000 UTC m=+2.941439443" lastFinishedPulling="2026-06-02 02:45:40.892958368 +0000 UTC m=+20.864558843" observedRunningTime="2026-06-02 02:45:41.802668995 +0000 UTC m=+21.774269481" watchObservedRunningTime="2026-06-02 02:45:41.802978361 +0000 UTC m=+21.774578846" Jun 02 02:45:41.819878 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:41.819847 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2zfx6" podStartSLOduration=8.670522465 podStartE2EDuration="21.819836564s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.972702059 +0000 UTC m=+2.944302525" lastFinishedPulling="2026-06-02 02:45:36.122016149 +0000 UTC m=+16.093616624" observedRunningTime="2026-06-02 02:45:41.819742147 +0000 UTC m=+21.791342628" watchObservedRunningTime="2026-06-02 02:45:41.819836564 +0000 UTC m=+21.791437048" Jun 02 02:45:42.271596 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.271573 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 02:45:42.447531 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.447366 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T02:45:42.271590595Z","UUID":"0a78e9db-6227-486b-8ab8-ac6e1b2892f8","Handler":null,"Name":"","Endpoint":""} Jun 02 02:45:42.448959 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.448911 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 02:45:42.448959 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.448938 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 02:45:42.506289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.506257 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:42.506289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.506270 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:42.506480 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:42.506379 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:42.506540 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:42.506476 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:42.726844 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.726770 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xqk88" event={"ID":"2b79b5d7-0ab0-4743-9c8a-c6593120483c","Type":"ContainerStarted","Data":"3d03bbfabdb13e60313c8c0643d44857d464ead38567ab2e9a9ed58698b21042"} Jun 02 02:45:42.728833 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.728799 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" event={"ID":"ad482e70-618c-45fa-83d5-d7ad477d7d50","Type":"ContainerStarted","Data":"0fe64fe430596e0c6bc93f0216cbce720fed8a3f5a4e162a54fe4430d0a4541c"} Jun 02 02:45:42.741943 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:42.741886 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xqk88" podStartSLOduration=4.8204280189999995 podStartE2EDuration="22.741874244s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.971459674 +0000 UTC m=+2.943060138" lastFinishedPulling="2026-06-02 02:45:40.892905894 +0000 UTC m=+20.864506363" observedRunningTime="2026-06-02 02:45:42.741316447 +0000 UTC m=+22.712916931" watchObservedRunningTime="2026-06-02 02:45:42.741874244 +0000 UTC m=+22.713474729" Jun 02 02:45:43.061159 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.061065 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:43.061844 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.061780 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:43.506376 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.506212 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:43.506512 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:43.506460 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:43.733317 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.733287 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" event={"ID":"ad482e70-618c-45fa-83d5-d7ad477d7d50","Type":"ContainerStarted","Data":"d4a4767971882e9e2d32d9d929a1f05109050ca18ea01aa82a8134fe1b22fac9"} Jun 02 02:45:43.733817 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.733793 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:43.734289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.734272 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2zfx6" Jun 02 02:45:43.751256 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:43.751223 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ff7c9" podStartSLOduration=3.421683396 podStartE2EDuration="23.751213037s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.975548296 +0000 UTC m=+2.947148764" lastFinishedPulling="2026-06-02 02:45:43.305077934 +0000 UTC m=+23.276678405" observedRunningTime="2026-06-02 02:45:43.750983346 +0000 UTC m=+23.722583830" watchObservedRunningTime="2026-06-02 02:45:43.751213037 +0000 UTC m=+23.722813522" Jun 02 02:45:44.506377 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:44.506339 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:44.506377 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:44.506343 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:44.506587 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:44.506468 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:44.506587 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:44.506570 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:44.737739 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:44.737714 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:45:44.738176 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:44.738045 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"b197a5cf1ae4d61eb8bff6018da5861f2969e5eb1407bfe97ee4f584d3b32fcf"} Jun 02 02:45:45.507031 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:45.506983 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:45.507226 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:45.507117 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:46.506617 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.506361 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:46.507349 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.506372 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:46.507349 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:46.506734 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:46.507349 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:46.506765 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:46.743753 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.743717 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="2eb1ed6f74f44c08257b9c6efadfebed5a38e08fc8c539f9dbe642b68a366dbb" exitCode=0 Jun 02 02:45:46.743923 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.743797 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"2eb1ed6f74f44c08257b9c6efadfebed5a38e08fc8c539f9dbe642b68a366dbb"} Jun 02 02:45:46.746821 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.746803 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:45:46.747201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.747178 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"eab72826e6f17f9b85e42e840ff36b0168dc83578a5092751c922b0be2db1898"} Jun 02 02:45:46.747486 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.747466 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:46.747570 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.747493 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:46.747635 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.747620 2563 scope.go:117] "RemoveContainer" containerID="26163de22c036f3786d3e32dc8e008935e31337b466fede7479f026b8c1bb778" Jun 02 02:45:46.762964 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.762945 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:46.763050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:46.763028 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:47.468452 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.468430 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:45:47.506201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.506174 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:47.506293 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:47.506266 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:47.754976 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.754886 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="f2d5ad78a5b3b6f314d45fb0052c7bdcb553d44a9b5cfd6c8f52bee134b7971d" exitCode=0 Jun 02 02:45:47.755426 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.754968 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"f2d5ad78a5b3b6f314d45fb0052c7bdcb553d44a9b5cfd6c8f52bee134b7971d"} Jun 02 02:45:47.758695 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.758677 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:45:47.759052 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.759002 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" event={"ID":"513f443e-a2f6-401d-a082-4222d02fdcd0","Type":"ContainerStarted","Data":"a267dde6309d29d72c3528a3e81a8a0fd00c4c955612849861e16938f162a24a"} Jun 02 02:45:47.829083 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:47.829038 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" podStartSLOduration=9.857814491 podStartE2EDuration="27.829023762s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.976483245 +0000 UTC m=+2.948083709" lastFinishedPulling="2026-06-02 02:45:40.947692516 +0000 UTC m=+20.919292980" observedRunningTime="2026-06-02 02:45:47.82858528 +0000 UTC m=+27.800185765" watchObservedRunningTime="2026-06-02 02:45:47.829023762 +0000 UTC m=+27.800624248" Jun 02 02:45:48.289894 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.289866 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vzm8k"] Jun 02 02:45:48.290048 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.289997 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:48.290105 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:48.290083 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:48.292945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.292918 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rml5c"] Jun 02 02:45:48.293125 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.293023 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:48.293201 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:48.293132 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:48.293731 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.293705 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qrl6f"] Jun 02 02:45:48.293826 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.293804 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:48.293916 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:48.293894 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:48.763019 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.762832 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="5606bf836f0110392d73f5a6f4eac463597e00d7a18b0cdc925e6511b5780119" exitCode=0 Jun 02 02:45:48.763019 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:48.762864 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"5606bf836f0110392d73f5a6f4eac463597e00d7a18b0cdc925e6511b5780119"} Jun 02 02:45:49.506305 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:49.506272 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:49.506475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:49.506317 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:49.506475 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:49.506428 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:49.506565 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:49.506537 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:50.507271 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:50.507240 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:50.507636 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:50.507334 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:51.506984 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:51.506937 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:51.507215 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:51.506937 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:51.507215 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:51.507084 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:51.507315 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:51.507208 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:52.506263 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:52.506169 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:52.506413 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:52.506298 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vzm8k" podUID="8849acd6-75bb-4056-abc8-b1d2f68de1e2" Jun 02 02:45:53.506122 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.506096 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:53.506695 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.506101 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:53.506695 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:53.506238 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qrl6f" podUID="d8a3dee4-d352-4d4a-a453-62cea3c0ce58" Jun 02 02:45:53.506695 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:53.506369 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:45:53.944854 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.944820 2563 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-19.ec2.internal" event="NodeReady" Jun 02 02:45:53.945031 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.944947 2563 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 02:45:53.988644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.988611 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2m228"] Jun 02 02:45:53.993831 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.993808 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q6mst"] Jun 02 02:45:53.993987 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.993965 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2m228" Jun 02 02:45:53.996411 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.996384 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 02:45:53.996526 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.996438 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5469w\"" Jun 02 02:45:53.996526 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.996438 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 02:45:53.998548 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:53.998526 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.000659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.000639 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 02:45:54.000769 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.000713 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 02:45:54.000769 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.000734 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcfjp\"" Jun 02 02:45:54.000883 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.000857 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 02:45:54.003032 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.002985 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2m228"] Jun 02 02:45:54.003806 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.003784 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6mst"] Jun 02 02:45:54.157796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157710 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-config-volume\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.157796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbh7c\" (UniqueName: \"kubernetes.io/projected/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-kube-api-access-rbh7c\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.158133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49chr\" (UniqueName: \"kubernetes.io/projected/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-kube-api-access-49chr\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.158133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157881 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.158133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-tmp-dir\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.158133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.157945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259109 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259028 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-config-volume\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259109 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259065 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rbh7c\" (UniqueName: \"kubernetes.io/projected/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-kube-api-access-rbh7c\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259109 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259106 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49chr\" (UniqueName: \"kubernetes.io/projected/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-kube-api-access-49chr\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.259392 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259155 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:54.259392 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259184 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.259392 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259210 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-tmp-dir\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259392 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259249 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259392 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259344 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259426 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:54.759406061 +0000 UTC m=+34.731006538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259352 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259532 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-tmp-dir\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259590 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:45:54.75957041 +0000 UTC m=+34.731170886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.259686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-config-volume\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.259706 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259709 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:54.259946 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.259765 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:26.259750605 +0000 UTC m=+66.231351071 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 02:45:54.272603 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.272566 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbh7c\" (UniqueName: \"kubernetes.io/projected/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-kube-api-access-rbh7c\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.272730 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.272620 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49chr\" (UniqueName: \"kubernetes.io/projected/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-kube-api-access-49chr\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.360220 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.360189 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:54.360376 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.360355 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 02:45:54.360438 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.360381 2563 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 02:45:54.360438 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.360395 2563 projected.go:194] Error preparing data for projected volume kube-api-access-rn4c6 for pod openshift-network-diagnostics/network-check-target-vzm8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:54.360523 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.360451 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6 podName:8849acd6-75bb-4056-abc8-b1d2f68de1e2 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:26.360437034 +0000 UTC m=+66.332037500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rn4c6" (UniqueName: "kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6") pod "network-check-target-vzm8k" (UID: "8849acd6-75bb-4056-abc8-b1d2f68de1e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 02:45:54.506670 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.506633 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:45:54.509467 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.509386 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 02:45:54.509467 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.509391 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 02:45:54.509644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.509397 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ldjq9\"" Jun 02 02:45:54.764691 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.764608 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:54.764855 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:54.764707 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:54.764855 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.764780 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:45:54.764855 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.764810 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:45:54.764855 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.764852 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:45:55.764835721 +0000 UTC m=+35.736436186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:45:54.765041 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:54.764867 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:55.764860954 +0000 UTC m=+35.736461417 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:45:55.506202 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.506170 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:55.506346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.506170 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:45:55.508997 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.508974 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 02:45:55.509991 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.509973 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgk2p\"" Jun 02 02:45:55.510105 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.510050 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 02:45:55.773081 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.772909 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:55.773226 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.773101 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:55.773226 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:55.773047 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:45:55.773226 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:55.773220 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:45:57.773205637 +0000 UTC m=+37.744806100 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:45:55.773355 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:55.773230 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:45:55.773355 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:55.773267 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:45:57.773255907 +0000 UTC m=+37.744856369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:45:55.780706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.780684 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="d368cf2d4c08605c65c60806c1ad0a8f012657325b7b89090e4333c28939c326" exitCode=0 Jun 02 02:45:55.780801 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:55.780737 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"d368cf2d4c08605c65c60806c1ad0a8f012657325b7b89090e4333c28939c326"} Jun 02 02:45:56.479332 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.479291 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:56.481475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.481450 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d8a3dee4-d352-4d4a-a453-62cea3c0ce58-original-pull-secret\") pod \"global-pull-secret-syncer-qrl6f\" (UID: \"d8a3dee4-d352-4d4a-a453-62cea3c0ce58\") " pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:56.715863 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.715831 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrl6f" Jun 02 02:45:56.785840 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.785745 2563 generic.go:358] "Generic (PLEG): container finished" podID="0110ac79-5ae3-4da0-9700-60cd268dc3a9" containerID="2cef5f6633c739c8da2dbb3bdf4b1b0616835b2626098a9405d292c6b0f73ae3" exitCode=0 Jun 02 02:45:56.785840 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.785804 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerDied","Data":"2cef5f6633c739c8da2dbb3bdf4b1b0616835b2626098a9405d292c6b0f73ae3"} Jun 02 02:45:56.856457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:56.856405 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qrl6f"] Jun 02 02:45:56.865354 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:45:56.865309 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a3dee4_d352_4d4a_a453_62cea3c0ce58.slice/crio-c94562e62a6c602eaabc1d7413b5d91c79a67ec8ee8fce4f054c521c9a6dafd9 WatchSource:0}: Error finding container c94562e62a6c602eaabc1d7413b5d91c79a67ec8ee8fce4f054c521c9a6dafd9: Status 404 returned error can't find the container with id c94562e62a6c602eaabc1d7413b5d91c79a67ec8ee8fce4f054c521c9a6dafd9 Jun 02 02:45:57.790237 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:57.789983 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:45:57.790659 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:57.790124 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:45:57.790659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:57.790343 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:45:57.790659 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:57.790387 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.790368227 +0000 UTC m=+41.761968694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:45:57.790659 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:57.790462 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:45:57.790659 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:45:57.790509 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:01.790493144 +0000 UTC m=+41.762093620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:45:57.791624 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:57.791598 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" event={"ID":"0110ac79-5ae3-4da0-9700-60cd268dc3a9","Type":"ContainerStarted","Data":"06d78b0be3a2be04b4b9d2c5a313a23362162a18c13691a288faf04328f1a166"} Jun 02 02:45:57.792630 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:57.792607 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qrl6f" event={"ID":"d8a3dee4-d352-4d4a-a453-62cea3c0ce58","Type":"ContainerStarted","Data":"c94562e62a6c602eaabc1d7413b5d91c79a67ec8ee8fce4f054c521c9a6dafd9"} Jun 02 02:45:57.822794 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:45:57.822743 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fhbxx" podStartSLOduration=5.486684984 podStartE2EDuration="37.82272773s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:45:22.964907093 +0000 UTC m=+2.936507563" lastFinishedPulling="2026-06-02 02:45:55.300949842 +0000 UTC m=+35.272550309" observedRunningTime="2026-06-02 02:45:57.821276 +0000 UTC m=+37.792876485" watchObservedRunningTime="2026-06-02 02:45:57.82272773 +0000 UTC m=+37.794328217" Jun 02 02:46:01.800991 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:01.800949 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qrl6f" event={"ID":"d8a3dee4-d352-4d4a-a453-62cea3c0ce58","Type":"ContainerStarted","Data":"ff3d277bc8c67a22fce481baa132705e559c202e623db7cf34afc6a801711047"} Jun 02 02:46:01.815621 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:01.815588 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:46:01.815740 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:01.815650 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:46:01.815789 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:01.815769 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:46:01.815826 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:01.815821 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:46:09.815804133 +0000 UTC m=+49.787404609 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:46:01.815881 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:01.815865 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:46:01.815930 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:01.815924 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:09.815905625 +0000 UTC m=+49.787506088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:46:01.816085 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:01.816047 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qrl6f" podStartSLOduration=33.714725991 podStartE2EDuration="37.816032694s" podCreationTimestamp="2026-06-02 02:45:24 +0000 UTC" firstStartedPulling="2026-06-02 02:45:56.86723043 +0000 UTC m=+36.838830896" lastFinishedPulling="2026-06-02 02:46:00.968537127 +0000 UTC m=+40.940137599" observedRunningTime="2026-06-02 02:46:01.815265493 +0000 UTC m=+41.786865988" watchObservedRunningTime="2026-06-02 02:46:01.816032694 +0000 UTC m=+41.787633181" Jun 02 02:46:09.862515 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:09.862476 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:46:09.862515 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:09.862518 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:46:09.863115 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:09.862614 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:46:09.863115 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:09.862631 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:46:09.863115 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:09.862661 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:46:25.862647636 +0000 UTC m=+65.834248105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:46:09.863115 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:09.862693 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:25.862676789 +0000 UTC m=+65.834277253 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:46:18.773317 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:18.773287 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hd4s2" Jun 02 02:46:25.870899 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:25.870860 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:46:25.870899 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:25.870908 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:46:25.871370 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:25.871015 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:46:25.871370 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:25.871017 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:46:25.871370 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:25.871077 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:46:57.871062862 +0000 UTC m=+97.842663325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:46:25.871370 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:25.871091 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:46:57.871085765 +0000 UTC m=+97.842686229 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:46:26.273216 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.273179 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:46:26.276379 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.276355 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 02:46:26.283562 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:26.283543 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 02:46:26.283653 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:26.283627 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:30.283606001 +0000 UTC m=+130.255206463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : secret "metrics-daemon-secret" not found Jun 02 02:46:26.373998 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.373965 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:46:26.376859 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.376839 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 02:46:26.387310 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.387285 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 02:46:26.399196 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.399167 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn4c6\" (UniqueName: \"kubernetes.io/projected/8849acd6-75bb-4056-abc8-b1d2f68de1e2-kube-api-access-rn4c6\") pod \"network-check-target-vzm8k\" (UID: \"8849acd6-75bb-4056-abc8-b1d2f68de1e2\") " pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:46:26.620108 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.620033 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ldjq9\"" Jun 02 02:46:26.628228 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.628211 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:46:26.762999 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.762963 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vzm8k"] Jun 02 02:46:26.766056 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:46:26.766028 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8849acd6_75bb_4056_abc8_b1d2f68de1e2.slice/crio-49d979f486c217352f880bfbc50fd944ef4908626d07da369ce4795ec3810a53 WatchSource:0}: Error finding container 49d979f486c217352f880bfbc50fd944ef4908626d07da369ce4795ec3810a53: Status 404 returned error can't find the container with id 49d979f486c217352f880bfbc50fd944ef4908626d07da369ce4795ec3810a53 Jun 02 02:46:26.846680 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:26.846643 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vzm8k" event={"ID":"8849acd6-75bb-4056-abc8-b1d2f68de1e2","Type":"ContainerStarted","Data":"49d979f486c217352f880bfbc50fd944ef4908626d07da369ce4795ec3810a53"} Jun 02 02:46:30.855409 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:30.855372 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vzm8k" event={"ID":"8849acd6-75bb-4056-abc8-b1d2f68de1e2","Type":"ContainerStarted","Data":"580af6fa5b18766d874cec3137c4334f30345af33c713209e94f955c368d467c"} Jun 02 02:46:30.855956 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:30.855583 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:46:30.871764 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:30.871723 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vzm8k" podStartSLOduration=67.704955547 podStartE2EDuration="1m10.871712962s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:46:26.767682907 +0000 UTC m=+66.739283370" lastFinishedPulling="2026-06-02 02:46:29.934440318 +0000 UTC m=+69.906040785" observedRunningTime="2026-06-02 02:46:30.871094922 +0000 UTC m=+70.842695407" watchObservedRunningTime="2026-06-02 02:46:30.871712962 +0000 UTC m=+70.843313446" Jun 02 02:46:57.890797 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:57.890755 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:46:57.891283 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:46:57.890807 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:46:57.891283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:57.890907 2563 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 02:46:57.891283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:57.890920 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 02:46:57.891283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:57.890983 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert podName:4ddc9c4c-0641-48c1-a4d6-ccb64393c120 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:01.890966159 +0000 UTC m=+161.862566622 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert") pod "ingress-canary-q6mst" (UID: "4ddc9c4c-0641-48c1-a4d6-ccb64393c120") : secret "canary-serving-cert" not found Jun 02 02:46:57.891283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:46:57.890998 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls podName:6270d4e2-89a3-42a9-8b55-483bfb2f8f9a nodeName:}" failed. No retries permitted until 2026-06-02 02:48:01.89099228 +0000 UTC m=+161.862592743 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls") pod "dns-default-2m228" (UID: "6270d4e2-89a3-42a9-8b55-483bfb2f8f9a") : secret "dns-default-metrics-tls" not found Jun 02 02:47:01.858868 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:01.858838 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vzm8k" Jun 02 02:47:30.304763 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:30.304721 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:47:30.305283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:30.304872 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 02:47:30.305283 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:30.304949 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs podName:70d6cad5-7d21-4aaf-b31a-9d1995494fd0 nodeName:}" failed. No retries permitted until 2026-06-02 02:49:32.304932125 +0000 UTC m=+252.276532592 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs") pod "network-metrics-daemon-rml5c" (UID: "70d6cad5-7d21-4aaf-b31a-9d1995494fd0") : secret "metrics-daemon-secret" not found Jun 02 02:47:45.594721 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.594686 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-wsz4b"] Jun 02 02:47:45.597697 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.597673 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.600044 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.600017 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 02:47:45.600044 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.600039 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kjjgj\"" Jun 02 02:47:45.600213 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.600078 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 02:47:45.601123 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.601104 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 02:47:45.601302 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.601287 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 02:47:45.608753 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.608727 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-wsz4b"] Jun 02 02:47:45.609406 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.609388 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 02:47:45.706453 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.706417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-config\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.706644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.706480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk77v\" (UniqueName: \"kubernetes.io/projected/9f97f813-3603-4f4f-884e-5a4492e8e0ef-kube-api-access-rk77v\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.706644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.706526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f97f813-3603-4f4f-884e-5a4492e8e0ef-serving-cert\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.706644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.706563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-trusted-ca\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.807784 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.807754 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rk77v\" (UniqueName: \"kubernetes.io/projected/9f97f813-3603-4f4f-884e-5a4492e8e0ef-kube-api-access-rk77v\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.807965 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.807797 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f97f813-3603-4f4f-884e-5a4492e8e0ef-serving-cert\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.807965 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.807843 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-trusted-ca\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.807965 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.807868 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-config\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.808655 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.808629 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-config\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.808655 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.808646 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f97f813-3603-4f4f-884e-5a4492e8e0ef-trusted-ca\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.810277 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.810256 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f97f813-3603-4f4f-884e-5a4492e8e0ef-serving-cert\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.816044 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.816025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk77v\" (UniqueName: \"kubernetes.io/projected/9f97f813-3603-4f4f-884e-5a4492e8e0ef-kube-api-access-rk77v\") pod \"console-operator-86d7c6cc77-wsz4b\" (UID: \"9f97f813-3603-4f4f-884e-5a4492e8e0ef\") " pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.892260 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.892226 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92"] Jun 02 02:47:45.895524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.895504 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-788bf58df5-dt6tw"] Jun 02 02:47:45.895681 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.895666 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:45.898089 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.898069 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:47:45.898236 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.898222 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:45.899433 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.899414 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 02:47:45.899567 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.899553 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7cqkn\"" Jun 02 02:47:45.899996 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.899981 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 02:47:45.900075 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.900053 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 02:47:45.900350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.900334 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 02 02:47:45.900684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.900670 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 02:47:45.900878 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.900862 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:45.902333 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.902313 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 02:47:45.902861 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.902837 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 02:47:45.903182 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.903136 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2hcks\"" Jun 02 02:47:45.903406 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.903390 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 02 02:47:45.904079 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.904062 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 02:47:45.905764 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.905747 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 02:47:45.906158 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.906131 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 02:47:45.907700 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.907681 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bqz7l\"" Jun 02 02:47:45.908029 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.907720 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:45.909674 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.909653 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92"] Jun 02 02:47:45.911324 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.911297 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-dt6tw"] Jun 02 02:47:45.911668 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.911650 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 02 02:47:45.913525 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.913504 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:47:45.914512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:45.914495 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 02:47:46.008724 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-tmp\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.008869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.008869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krfg5\" (UniqueName: \"kubernetes.io/projected/6f60493a-6291-4ded-971b-8c016d0fc23e-kube-api-access-krfg5\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.008869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.008869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008865 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.009050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008887 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8glzc\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.009050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec240069-31bf-4333-b5c0-90829b8c73f0-serving-cert\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.009050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.008927 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.009050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009012 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-service-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.009200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.009200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009088 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f60493a-6291-4ded-971b-8c016d0fc23e-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.009200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98jtk\" (UniqueName: \"kubernetes.io/projected/ec240069-31bf-4333-b5c0-90829b8c73f0-kube-api-access-98jtk\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.009200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009170 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.009390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009198 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.009390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f60493a-6291-4ded-971b-8c016d0fc23e-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.009390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009251 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-snapshots\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.009390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.009345 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.030920 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.030890 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-wsz4b"] Jun 02 02:47:46.034409 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:47:46.034378 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f97f813_3603_4f4f_884e_5a4492e8e0ef.slice/crio-abcbbf28a69ced97d3ee7abe4f36c89addd437894a35025af05a339b30b44680 WatchSource:0}: Error finding container abcbbf28a69ced97d3ee7abe4f36c89addd437894a35025af05a339b30b44680: Status 404 returned error can't find the container with id abcbbf28a69ced97d3ee7abe4f36c89addd437894a35025af05a339b30b44680 Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111108 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-tmp\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111199 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111242 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-krfg5\" (UniqueName: \"kubernetes.io/projected/6f60493a-6291-4ded-971b-8c016d0fc23e-kube-api-access-krfg5\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111290 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111329 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111365 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8glzc\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111405 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec240069-31bf-4333-b5c0-90829b8c73f0-serving-cert\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111438 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111476 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-service-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111509 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111546 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f60493a-6291-4ded-971b-8c016d0fc23e-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111576 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-98jtk\" (UniqueName: \"kubernetes.io/projected/ec240069-31bf-4333-b5c0-90829b8c73f0-kube-api-access-98jtk\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111616 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111650 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.113958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111681 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f60493a-6291-4ded-971b-8c016d0fc23e-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111750 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-snapshots\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.111796 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.112497 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.112869 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-tmp\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.114445 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-service-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.114814 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.114686 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.115132 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.115057 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6f60493a-6291-4ded-971b-8c016d0fc23e-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.115132 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.115084 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ec240069-31bf-4333-b5c0-90829b8c73f0-snapshots\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.115270 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.115221 2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 02:47:46.115270 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.115238 2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666d6d74cc-zltsz: secret "image-registry-tls" not found Jun 02 02:47:46.115373 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.115301 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls podName:a91b6a74-7d69-4950-9ada-a6d91ac2ea22 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:46.615277864 +0000 UTC m=+146.586878346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls") pod "image-registry-666d6d74cc-zltsz" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22") : secret "image-registry-tls" not found Jun 02 02:47:46.115552 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.115531 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec240069-31bf-4333-b5c0-90829b8c73f0-trusted-ca-bundle\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.115835 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.115758 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.116403 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.116372 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.116602 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.116582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6f60493a-6291-4ded-971b-8c016d0fc23e-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.117327 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.117297 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ec240069-31bf-4333-b5c0-90829b8c73f0-serving-cert\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.117606 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.117589 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.123465 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.123441 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.124063 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.124044 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8glzc\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.124129 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.124080 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krfg5\" (UniqueName: \"kubernetes.io/projected/6f60493a-6291-4ded-971b-8c016d0fc23e-kube-api-access-krfg5\") pod \"kube-storage-version-migrator-operator-96ddc787c-2rd92\" (UID: \"6f60493a-6291-4ded-971b-8c016d0fc23e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.124397 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.124380 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98jtk\" (UniqueName: \"kubernetes.io/projected/ec240069-31bf-4333-b5c0-90829b8c73f0-kube-api-access-98jtk\") pod \"insights-operator-788bf58df5-dt6tw\" (UID: \"ec240069-31bf-4333-b5c0-90829b8c73f0\") " pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.205871 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.205787 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" Jun 02 02:47:46.214657 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.214635 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" Jun 02 02:47:46.328501 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.328471 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92"] Jun 02 02:47:46.331867 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:47:46.331832 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f60493a_6291_4ded_971b_8c016d0fc23e.slice/crio-c82d542f1bf39e7addf41dacd3ab80556aa62c137cf6e9c8ce227df7f30f00a5 WatchSource:0}: Error finding container c82d542f1bf39e7addf41dacd3ab80556aa62c137cf6e9c8ce227df7f30f00a5: Status 404 returned error can't find the container with id c82d542f1bf39e7addf41dacd3ab80556aa62c137cf6e9c8ce227df7f30f00a5 Jun 02 02:47:46.348760 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.348734 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-788bf58df5-dt6tw"] Jun 02 02:47:46.351418 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:47:46.351394 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec240069_31bf_4333_b5c0_90829b8c73f0.slice/crio-e3b8882ac7de0e2733786aa492f941dbc5cbace4f26b1e8009f7d48e9d4cac97 WatchSource:0}: Error finding container e3b8882ac7de0e2733786aa492f941dbc5cbace4f26b1e8009f7d48e9d4cac97: Status 404 returned error can't find the container with id e3b8882ac7de0e2733786aa492f941dbc5cbace4f26b1e8009f7d48e9d4cac97 Jun 02 02:47:46.616529 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.616448 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:46.616947 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.616594 2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 02:47:46.616947 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.616609 2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666d6d74cc-zltsz: secret "image-registry-tls" not found Jun 02 02:47:46.616947 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:46.616675 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls podName:a91b6a74-7d69-4950-9ada-a6d91ac2ea22 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:47.616654935 +0000 UTC m=+147.588255412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls") pod "image-registry-666d6d74cc-zltsz" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22") : secret "image-registry-tls" not found Jun 02 02:47:46.997417 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.997364 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" event={"ID":"6f60493a-6291-4ded-971b-8c016d0fc23e","Type":"ContainerStarted","Data":"c82d542f1bf39e7addf41dacd3ab80556aa62c137cf6e9c8ce227df7f30f00a5"} Jun 02 02:47:46.999208 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:46.999109 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" event={"ID":"ec240069-31bf-4333-b5c0-90829b8c73f0","Type":"ContainerStarted","Data":"e3b8882ac7de0e2733786aa492f941dbc5cbace4f26b1e8009f7d48e9d4cac97"} Jun 02 02:47:47.001160 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:47.001116 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" event={"ID":"9f97f813-3603-4f4f-884e-5a4492e8e0ef","Type":"ContainerStarted","Data":"abcbbf28a69ced97d3ee7abe4f36c89addd437894a35025af05a339b30b44680"} Jun 02 02:47:47.625017 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:47.624977 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:47.625424 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:47.625098 2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 02:47:47.625424 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:47.625124 2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666d6d74cc-zltsz: secret "image-registry-tls" not found Jun 02 02:47:47.625424 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:47.625205 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls podName:a91b6a74-7d69-4950-9ada-a6d91ac2ea22 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:49.625188782 +0000 UTC m=+149.596789244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls") pod "image-registry-666d6d74cc-zltsz" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22") : secret "image-registry-tls" not found Jun 02 02:47:49.640046 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:49.640007 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:49.640428 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:49.640167 2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 02:47:49.640428 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:49.640191 2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666d6d74cc-zltsz: secret "image-registry-tls" not found Jun 02 02:47:49.640428 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:49.640245 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls podName:a91b6a74-7d69-4950-9ada-a6d91ac2ea22 nodeName:}" failed. No retries permitted until 2026-06-02 02:47:53.640229438 +0000 UTC m=+153.611829906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls") pod "image-registry-666d6d74cc-zltsz" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22") : secret "image-registry-tls" not found Jun 02 02:47:50.008817 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.008724 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" event={"ID":"6f60493a-6291-4ded-971b-8c016d0fc23e","Type":"ContainerStarted","Data":"f18a8b9426304da29a2bddc45af7782bdcdad3e9db781dd7b06b74c870295d4c"} Jun 02 02:47:50.010107 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.010075 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" event={"ID":"ec240069-31bf-4333-b5c0-90829b8c73f0","Type":"ContainerStarted","Data":"b9eab5e68cc8a6c0dc72a5913d03dda47840830c5554bedd09f1b1c9218ca99d"} Jun 02 02:47:50.011592 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.011571 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/0.log" Jun 02 02:47:50.011695 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.011614 2563 generic.go:358] "Generic (PLEG): container finished" podID="9f97f813-3603-4f4f-884e-5a4492e8e0ef" containerID="23546b1a9ec12c9431d006c367ea5f08bdae107326b37c6e23199801dc8d8ec3" exitCode=255 Jun 02 02:47:50.011695 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.011660 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" event={"ID":"9f97f813-3603-4f4f-884e-5a4492e8e0ef","Type":"ContainerDied","Data":"23546b1a9ec12c9431d006c367ea5f08bdae107326b37c6e23199801dc8d8ec3"} Jun 02 02:47:50.011873 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.011857 2563 scope.go:117] "RemoveContainer" containerID="23546b1a9ec12c9431d006c367ea5f08bdae107326b37c6e23199801dc8d8ec3" Jun 02 02:47:50.024266 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.024206 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" podStartSLOduration=2.313943923 podStartE2EDuration="5.024189903s" podCreationTimestamp="2026-06-02 02:47:45 +0000 UTC" firstStartedPulling="2026-06-02 02:47:46.333832391 +0000 UTC m=+146.305432854" lastFinishedPulling="2026-06-02 02:47:49.04407837 +0000 UTC m=+149.015678834" observedRunningTime="2026-06-02 02:47:50.024014884 +0000 UTC m=+149.995615381" watchObservedRunningTime="2026-06-02 02:47:50.024189903 +0000 UTC m=+149.995790389" Jun 02 02:47:50.057611 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.057558 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" podStartSLOduration=2.3733109199999998 podStartE2EDuration="5.05754044s" podCreationTimestamp="2026-06-02 02:47:45 +0000 UTC" firstStartedPulling="2026-06-02 02:47:46.353888453 +0000 UTC m=+146.325488928" lastFinishedPulling="2026-06-02 02:47:49.038117974 +0000 UTC m=+149.009718448" observedRunningTime="2026-06-02 02:47:50.057241397 +0000 UTC m=+150.028841905" watchObservedRunningTime="2026-06-02 02:47:50.05754044 +0000 UTC m=+150.029140926" Jun 02 02:47:50.907995 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.907963 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2"] Jun 02 02:47:50.911016 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.910995 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" Jun 02 02:47:50.913666 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.913646 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 02 02:47:50.913801 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.913650 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 02 02:47:50.914789 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.914771 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-r858n\"" Jun 02 02:47:50.918488 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:50.918461 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2"] Jun 02 02:47:51.015812 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.015786 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:47:51.016158 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.016132 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/0.log" Jun 02 02:47:51.016246 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.016179 2563 generic.go:358] "Generic (PLEG): container finished" podID="9f97f813-3603-4f4f-884e-5a4492e8e0ef" containerID="d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605" exitCode=255 Jun 02 02:47:51.016246 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.016212 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" event={"ID":"9f97f813-3603-4f4f-884e-5a4492e8e0ef","Type":"ContainerDied","Data":"d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605"} Jun 02 02:47:51.016321 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.016265 2563 scope.go:117] "RemoveContainer" containerID="23546b1a9ec12c9431d006c367ea5f08bdae107326b37c6e23199801dc8d8ec3" Jun 02 02:47:51.016524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.016504 2563 scope.go:117] "RemoveContainer" containerID="d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605" Jun 02 02:47:51.016751 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:51.016721 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-wsz4b_openshift-console-operator(9f97f813-3603-4f4f-884e-5a4492e8e0ef)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" podUID="9f97f813-3603-4f4f-884e-5a4492e8e0ef" Jun 02 02:47:51.051810 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.051783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzdfc\" (UniqueName: \"kubernetes.io/projected/25221423-6136-4101-ba83-4345e9fc0ea7-kube-api-access-hzdfc\") pod \"migrator-96c6f749-lspf2\" (UID: \"25221423-6136-4101-ba83-4345e9fc0ea7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" Jun 02 02:47:51.152572 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.152538 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hzdfc\" (UniqueName: \"kubernetes.io/projected/25221423-6136-4101-ba83-4345e9fc0ea7-kube-api-access-hzdfc\") pod \"migrator-96c6f749-lspf2\" (UID: \"25221423-6136-4101-ba83-4345e9fc0ea7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" Jun 02 02:47:51.160882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.160826 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzdfc\" (UniqueName: \"kubernetes.io/projected/25221423-6136-4101-ba83-4345e9fc0ea7-kube-api-access-hzdfc\") pod \"migrator-96c6f749-lspf2\" (UID: \"25221423-6136-4101-ba83-4345e9fc0ea7\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" Jun 02 02:47:51.220820 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.220783 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" Jun 02 02:47:51.335694 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:51.335658 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2"] Jun 02 02:47:51.339776 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:47:51.339749 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25221423_6136_4101_ba83_4345e9fc0ea7.slice/crio-df4ee1a399e91dfbdb8888361833e812ad5de01e7ecc6003e7204072a346bf77 WatchSource:0}: Error finding container df4ee1a399e91dfbdb8888361833e812ad5de01e7ecc6003e7204072a346bf77: Status 404 returned error can't find the container with id df4ee1a399e91dfbdb8888361833e812ad5de01e7ecc6003e7204072a346bf77 Jun 02 02:47:52.021315 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:52.021284 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:47:52.021866 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:52.021843 2563 scope.go:117] "RemoveContainer" containerID="d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605" Jun 02 02:47:52.022487 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:52.022460 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-wsz4b_openshift-console-operator(9f97f813-3603-4f4f-884e-5a4492e8e0ef)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" podUID="9f97f813-3603-4f4f-884e-5a4492e8e0ef" Jun 02 02:47:52.023121 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:52.023102 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" event={"ID":"25221423-6136-4101-ba83-4345e9fc0ea7","Type":"ContainerStarted","Data":"df4ee1a399e91dfbdb8888361833e812ad5de01e7ecc6003e7204072a346bf77"} Jun 02 02:47:52.765654 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:52.765635 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hwplc_bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c/dns-node-resolver/0.log" Jun 02 02:47:53.028372 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:53.027180 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" event={"ID":"25221423-6136-4101-ba83-4345e9fc0ea7","Type":"ContainerStarted","Data":"960f89134e73b166aff104c07311cec62547d77f64720494d25fa645791562e4"} Jun 02 02:47:53.028372 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:53.027224 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" event={"ID":"25221423-6136-4101-ba83-4345e9fc0ea7","Type":"ContainerStarted","Data":"6dd53132c08e0aa2df6941c85c678523b0b1d54aa2b316f85288b5ccf2c82f29"} Jun 02 02:47:53.046039 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:53.045985 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-lspf2" podStartSLOduration=1.738173252 podStartE2EDuration="3.045972248s" podCreationTimestamp="2026-06-02 02:47:50 +0000 UTC" firstStartedPulling="2026-06-02 02:47:51.341679774 +0000 UTC m=+151.313280240" lastFinishedPulling="2026-06-02 02:47:52.649478773 +0000 UTC m=+152.621079236" observedRunningTime="2026-06-02 02:47:53.04470292 +0000 UTC m=+153.016303404" watchObservedRunningTime="2026-06-02 02:47:53.045972248 +0000 UTC m=+153.017572733" Jun 02 02:47:53.566781 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:53.566755 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gd2qn_cb382c8c-5050-49a7-a71f-75325166bbc9/node-ca/0.log" Jun 02 02:47:53.668289 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:53.668251 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:47:53.668471 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:53.668426 2563 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 02:47:53.668471 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:53.668448 2563 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666d6d74cc-zltsz: secret "image-registry-tls" not found Jun 02 02:47:53.668577 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:53.668517 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls podName:a91b6a74-7d69-4950-9ada-a6d91ac2ea22 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:01.668497077 +0000 UTC m=+161.640097541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls") pod "image-registry-666d6d74cc-zltsz" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22") : secret "image-registry-tls" not found Jun 02 02:47:55.908677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:55.908625 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:55.908677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:55.908683 2563 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:47:55.909119 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:55.909024 2563 scope.go:117] "RemoveContainer" containerID="d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605" Jun 02 02:47:55.909225 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:55.909208 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-wsz4b_openshift-console-operator(9f97f813-3603-4f4f-884e-5a4492e8e0ef)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" podUID="9f97f813-3603-4f4f-884e-5a4492e8e0ef" Jun 02 02:47:57.007467 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:57.007417 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2m228" podUID="6270d4e2-89a3-42a9-8b55-483bfb2f8f9a" Jun 02 02:47:57.013070 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:57.013040 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-q6mst" podUID="4ddc9c4c-0641-48c1-a4d6-ccb64393c120" Jun 02 02:47:57.037031 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:57.037006 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2m228" Jun 02 02:47:57.037140 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:47:57.037006 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:47:58.521618 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:47:58.521580 2563 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rml5c" podUID="70d6cad5-7d21-4aaf-b31a-9d1995494fd0" Jun 02 02:48:01.729577 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.729533 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:01.731818 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.731798 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"image-registry-666d6d74cc-zltsz\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:01.829608 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.829577 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:01.930772 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.930717 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:48:01.930896 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.930800 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:48:01.933018 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.932994 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6270d4e2-89a3-42a9-8b55-483bfb2f8f9a-metrics-tls\") pod \"dns-default-2m228\" (UID: \"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a\") " pod="openshift-dns/dns-default-2m228" Jun 02 02:48:01.933125 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.933041 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ddc9c4c-0641-48c1-a4d6-ccb64393c120-cert\") pod \"ingress-canary-q6mst\" (UID: \"4ddc9c4c-0641-48c1-a4d6-ccb64393c120\") " pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:48:01.949652 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:01.949624 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:48:01.953178 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:01.953138 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91b6a74_7d69_4950_9ada_a6d91ac2ea22.slice/crio-46f6b4b8750acea702ca0d8b4ec6ebbabdd91dbf99b2d2fba9d6b38199937e33 WatchSource:0}: Error finding container 46f6b4b8750acea702ca0d8b4ec6ebbabdd91dbf99b2d2fba9d6b38199937e33: Status 404 returned error can't find the container with id 46f6b4b8750acea702ca0d8b4ec6ebbabdd91dbf99b2d2fba9d6b38199937e33 Jun 02 02:48:02.052136 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.052103 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" event={"ID":"a91b6a74-7d69-4950-9ada-a6d91ac2ea22","Type":"ContainerStarted","Data":"9449feff78979d7a9c3858326e0d967cfdb966b84987745b65a83ebabeda249b"} Jun 02 02:48:02.052294 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.052138 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" event={"ID":"a91b6a74-7d69-4950-9ada-a6d91ac2ea22","Type":"ContainerStarted","Data":"46f6b4b8750acea702ca0d8b4ec6ebbabdd91dbf99b2d2fba9d6b38199937e33"} Jun 02 02:48:02.052294 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.052192 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:02.071863 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.071814 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" podStartSLOduration=17.071798588 podStartE2EDuration="17.071798588s" podCreationTimestamp="2026-06-02 02:47:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:48:02.070932785 +0000 UTC m=+162.042533270" watchObservedRunningTime="2026-06-02 02:48:02.071798588 +0000 UTC m=+162.043399087" Jun 02 02:48:02.140074 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.140042 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcfjp\"" Jun 02 02:48:02.140074 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.140051 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5469w\"" Jun 02 02:48:02.147710 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.147684 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q6mst" Jun 02 02:48:02.147710 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.147703 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2m228" Jun 02 02:48:02.279208 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.279181 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2m228"] Jun 02 02:48:02.281701 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:02.281667 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6270d4e2_89a3_42a9_8b55_483bfb2f8f9a.slice/crio-def56812a1e3be987ea55b7d9f99fa78555e85bc8db20a73b0f86789aff2fdda WatchSource:0}: Error finding container def56812a1e3be987ea55b7d9f99fa78555e85bc8db20a73b0f86789aff2fdda: Status 404 returned error can't find the container with id def56812a1e3be987ea55b7d9f99fa78555e85bc8db20a73b0f86789aff2fdda Jun 02 02:48:02.293068 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:02.293043 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q6mst"] Jun 02 02:48:02.296602 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:02.296580 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddc9c4c_0641_48c1_a4d6_ccb64393c120.slice/crio-70ab661ab4133995b8440e370f914dbc8a93c12c93388616947f8c1437ae9e6f WatchSource:0}: Error finding container 70ab661ab4133995b8440e370f914dbc8a93c12c93388616947f8c1437ae9e6f: Status 404 returned error can't find the container with id 70ab661ab4133995b8440e370f914dbc8a93c12c93388616947f8c1437ae9e6f Jun 02 02:48:03.056066 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:03.056002 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2m228" event={"ID":"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a","Type":"ContainerStarted","Data":"def56812a1e3be987ea55b7d9f99fa78555e85bc8db20a73b0f86789aff2fdda"} Jun 02 02:48:03.057702 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:03.057671 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6mst" event={"ID":"4ddc9c4c-0641-48c1-a4d6-ccb64393c120","Type":"ContainerStarted","Data":"70ab661ab4133995b8440e370f914dbc8a93c12c93388616947f8c1437ae9e6f"} Jun 02 02:48:05.063130 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.063099 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2m228" event={"ID":"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a","Type":"ContainerStarted","Data":"a355c2c337e3ba212f3445bad41b93879e39f2b8fe323caa719d468aed0e58e0"} Jun 02 02:48:05.063130 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.063132 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2m228" event={"ID":"6270d4e2-89a3-42a9-8b55-483bfb2f8f9a","Type":"ContainerStarted","Data":"0c72a6229274dbad93aeaed07cd5fd2acb6e58b01c3cf32ae6663f4c820a780b"} Jun 02 02:48:05.063616 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.063252 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2m228" Jun 02 02:48:05.064360 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.064340 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q6mst" event={"ID":"4ddc9c4c-0641-48c1-a4d6-ccb64393c120","Type":"ContainerStarted","Data":"6f924197840a8c7334692ecb1397137fddf8006cae5c4780415151d8de502625"} Jun 02 02:48:05.082080 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.079613 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-2m228" podStartSLOduration=130.090727138 podStartE2EDuration="2m12.079591444s" podCreationTimestamp="2026-06-02 02:45:53 +0000 UTC" firstStartedPulling="2026-06-02 02:48:02.284177416 +0000 UTC m=+162.255777880" lastFinishedPulling="2026-06-02 02:48:04.273041712 +0000 UTC m=+164.244642186" observedRunningTime="2026-06-02 02:48:05.077978897 +0000 UTC m=+165.049579386" watchObservedRunningTime="2026-06-02 02:48:05.079591444 +0000 UTC m=+165.051191930" Jun 02 02:48:05.094940 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:05.094902 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q6mst" podStartSLOduration=130.117203914 podStartE2EDuration="2m12.094886005s" podCreationTimestamp="2026-06-02 02:45:53 +0000 UTC" firstStartedPulling="2026-06-02 02:48:02.298615161 +0000 UTC m=+162.270215624" lastFinishedPulling="2026-06-02 02:48:04.276297252 +0000 UTC m=+164.247897715" observedRunningTime="2026-06-02 02:48:05.094457195 +0000 UTC m=+165.066057681" watchObservedRunningTime="2026-06-02 02:48:05.094886005 +0000 UTC m=+165.066486490" Jun 02 02:48:09.507038 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:09.506956 2563 scope.go:117] "RemoveContainer" containerID="d8ef57e9d9f6f3f0178b02fce77abd205eec8f740f3f284b8efb82bc4cc59605" Jun 02 02:48:10.078662 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:10.078623 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:48:10.078834 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:10.078711 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" event={"ID":"9f97f813-3603-4f4f-884e-5a4492e8e0ef","Type":"ContainerStarted","Data":"abd6db242495f592fa00b7c22095ab8a2df7def8dea4176767ce3a852b171688"} Jun 02 02:48:10.079527 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:10.079505 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:48:10.084238 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:10.084218 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" Jun 02 02:48:10.102570 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:10.102521 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-wsz4b" podStartSLOduration=22.098358993 podStartE2EDuration="25.102506187s" podCreationTimestamp="2026-06-02 02:47:45 +0000 UTC" firstStartedPulling="2026-06-02 02:47:46.036140418 +0000 UTC m=+146.007740881" lastFinishedPulling="2026-06-02 02:47:49.040287598 +0000 UTC m=+149.011888075" observedRunningTime="2026-06-02 02:48:10.101709474 +0000 UTC m=+170.073309971" watchObservedRunningTime="2026-06-02 02:48:10.102506187 +0000 UTC m=+170.074106703" Jun 02 02:48:12.506976 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:12.506945 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:48:14.866133 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.866068 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zqk28"] Jun 02 02:48:14.869586 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.869570 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.872361 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.872334 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 02:48:14.872471 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.872445 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 02:48:14.873491 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.873474 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2sb7h\"" Jun 02 02:48:14.879077 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.879056 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqk28"] Jun 02 02:48:14.925990 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.925961 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42649385-dcf3-44aa-8911-0232b0e26ac5-data-volume\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.926119 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.926000 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.926119 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.926020 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42649385-dcf3-44aa-8911-0232b0e26ac5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.926119 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.926067 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42649385-dcf3-44aa-8911-0232b0e26ac5-crio-socket\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.926258 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.926118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd8qc\" (UniqueName: \"kubernetes.io/projected/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-api-access-gd8qc\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:14.928505 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.928482 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-lx7qk"] Jun 02 02:48:14.931510 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.931490 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:14.932351 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.932329 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:48:14.935659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.935625 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 02:48:14.935659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.935646 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-h5khn\"" Jun 02 02:48:14.935805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.935752 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 02:48:14.942995 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:14.942974 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-lx7qk"] Jun 02 02:48:15.026930 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.026907 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6d8z\" (UniqueName: \"kubernetes.io/projected/dd45154e-530c-4be5-9cca-37172cf641f6-kube-api-access-n6d8z\") pod \"downloads-c65b8dc4b-lx7qk\" (UID: \"dd45154e-530c-4be5-9cca-37172cf641f6\") " pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:15.027051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.026947 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42649385-dcf3-44aa-8911-0232b0e26ac5-crio-socket\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.026985 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gd8qc\" (UniqueName: \"kubernetes.io/projected/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-api-access-gd8qc\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027005 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42649385-dcf3-44aa-8911-0232b0e26ac5-data-volume\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027038 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027051 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027035 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42649385-dcf3-44aa-8911-0232b0e26ac5-crio-socket\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027056 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42649385-dcf3-44aa-8911-0232b0e26ac5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027449 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42649385-dcf3-44aa-8911-0232b0e26ac5-data-volume\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.027572 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.027558 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.029455 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.029434 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42649385-dcf3-44aa-8911-0232b0e26ac5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.037259 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.037241 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd8qc\" (UniqueName: \"kubernetes.io/projected/42649385-dcf3-44aa-8911-0232b0e26ac5-kube-api-access-gd8qc\") pod \"insights-runtime-extractor-zqk28\" (UID: \"42649385-dcf3-44aa-8911-0232b0e26ac5\") " pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.069066 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.069042 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2m228" Jun 02 02:48:15.127451 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.127421 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n6d8z\" (UniqueName: \"kubernetes.io/projected/dd45154e-530c-4be5-9cca-37172cf641f6-kube-api-access-n6d8z\") pod \"downloads-c65b8dc4b-lx7qk\" (UID: \"dd45154e-530c-4be5-9cca-37172cf641f6\") " pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:15.137011 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.136984 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6d8z\" (UniqueName: \"kubernetes.io/projected/dd45154e-530c-4be5-9cca-37172cf641f6-kube-api-access-n6d8z\") pod \"downloads-c65b8dc4b-lx7qk\" (UID: \"dd45154e-530c-4be5-9cca-37172cf641f6\") " pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:15.179504 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.179480 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zqk28" Jun 02 02:48:15.243660 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.243465 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:15.310966 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.310914 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zqk28"] Jun 02 02:48:15.315708 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:15.315682 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42649385_dcf3_44aa_8911_0232b0e26ac5.slice/crio-ea050364cad058021f7f74184d9b7db8e2bcdad2c41d4d6958ce8c67eec9a52b WatchSource:0}: Error finding container ea050364cad058021f7f74184d9b7db8e2bcdad2c41d4d6958ce8c67eec9a52b: Status 404 returned error can't find the container with id ea050364cad058021f7f74184d9b7db8e2bcdad2c41d4d6958ce8c67eec9a52b Jun 02 02:48:15.369280 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:15.369260 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-lx7qk"] Jun 02 02:48:15.371673 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:15.371649 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd45154e_530c_4be5_9cca_37172cf641f6.slice/crio-1e2ff565e22fe27d71cfd4ba77eba2bfb86602d791e8115f380e2b096fcdff53 WatchSource:0}: Error finding container 1e2ff565e22fe27d71cfd4ba77eba2bfb86602d791e8115f380e2b096fcdff53: Status 404 returned error can't find the container with id 1e2ff565e22fe27d71cfd4ba77eba2bfb86602d791e8115f380e2b096fcdff53 Jun 02 02:48:16.100799 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:16.100765 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqk28" event={"ID":"42649385-dcf3-44aa-8911-0232b0e26ac5","Type":"ContainerStarted","Data":"aba2ba08aa701c73a63ce5941ef7330cd8bee54d50ae8d4bb58d309b5db03312"} Jun 02 02:48:16.101209 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:16.100806 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqk28" event={"ID":"42649385-dcf3-44aa-8911-0232b0e26ac5","Type":"ContainerStarted","Data":"ea050364cad058021f7f74184d9b7db8e2bcdad2c41d4d6958ce8c67eec9a52b"} Jun 02 02:48:16.101808 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:16.101779 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-lx7qk" event={"ID":"dd45154e-530c-4be5-9cca-37172cf641f6","Type":"ContainerStarted","Data":"1e2ff565e22fe27d71cfd4ba77eba2bfb86602d791e8115f380e2b096fcdff53"} Jun 02 02:48:17.106617 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.106572 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqk28" event={"ID":"42649385-dcf3-44aa-8911-0232b0e26ac5","Type":"ContainerStarted","Data":"9234bb1dedaa019fa33bffa81f7fb39d80a713caf27fdfdf44bfa8c15273e59c"} Jun 02 02:48:17.844780 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.844745 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:48:17.847739 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.847712 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.850528 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.850505 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 02:48:17.850622 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.850508 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 02:48:17.850760 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.850745 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 02:48:17.851635 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.851618 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 02:48:17.851744 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.851712 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rcdq4\"" Jun 02 02:48:17.851912 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.851888 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 02:48:17.859242 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.859220 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:48:17.951316 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbsbm\" (UniqueName: \"kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.951444 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951330 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.951444 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.951543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.951543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951492 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:17.951622 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:17.951543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052738 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052708 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rbsbm\" (UniqueName: \"kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052738 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052753 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052971 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052789 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052971 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052821 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052971 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052857 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.052971 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.052904 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.053623 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.053572 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.053744 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.053619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.053744 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.053639 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.055756 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.055735 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.055847 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.055826 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.064137 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.064115 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbsbm\" (UniqueName: \"kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm\") pod \"console-656b5b9df7-cjbtx\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.110869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.110808 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zqk28" event={"ID":"42649385-dcf3-44aa-8911-0232b0e26ac5","Type":"ContainerStarted","Data":"17ac412fd512fcaf3d1520afdbe0cf0751226b26aa8d1b57fcfad4f0ecb0e356"} Jun 02 02:48:18.132596 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.132558 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zqk28" podStartSLOduration=1.611928032 podStartE2EDuration="4.132547014s" podCreationTimestamp="2026-06-02 02:48:14 +0000 UTC" firstStartedPulling="2026-06-02 02:48:15.37029131 +0000 UTC m=+175.341891773" lastFinishedPulling="2026-06-02 02:48:17.890910292 +0000 UTC m=+177.862510755" observedRunningTime="2026-06-02 02:48:18.131917868 +0000 UTC m=+178.103518354" watchObservedRunningTime="2026-06-02 02:48:18.132547014 +0000 UTC m=+178.104147498" Jun 02 02:48:18.158317 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.158297 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:18.294706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:18.294676 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:48:18.297602 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:18.297577 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0feceb_2fc2_4dcf_9a0c_707bdf645a4e.slice/crio-73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d WatchSource:0}: Error finding container 73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d: Status 404 returned error can't find the container with id 73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d Jun 02 02:48:19.114797 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:19.114758 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b5b9df7-cjbtx" event={"ID":"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e","Type":"ContainerStarted","Data":"73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d"} Jun 02 02:48:19.993660 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:19.993541 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc"] Jun 02 02:48:19.997497 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:19.997477 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:20.000359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.000335 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-h46kk\"" Jun 02 02:48:20.000630 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.000614 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 02 02:48:20.009992 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.009953 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc"] Jun 02 02:48:20.073318 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.073289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-6brbc\" (UID: \"a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:20.174335 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.174255 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-6brbc\" (UID: \"a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:20.177562 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.177467 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-6brbc\" (UID: \"a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:20.310344 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:20.310268 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:21.628998 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:21.628947 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc"] Jun 02 02:48:21.633577 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:21.633548 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3043791_f4bb_4ebe_8b5e_5dd7e3c2fe09.slice/crio-d53f83f810c57b1da01c3d5ab58c39b8f6ea46795457f2022f8f8ebd22fa5ab9 WatchSource:0}: Error finding container d53f83f810c57b1da01c3d5ab58c39b8f6ea46795457f2022f8f8ebd22fa5ab9: Status 404 returned error can't find the container with id d53f83f810c57b1da01c3d5ab58c39b8f6ea46795457f2022f8f8ebd22fa5ab9 Jun 02 02:48:22.129940 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:22.129895 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" event={"ID":"a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09","Type":"ContainerStarted","Data":"d53f83f810c57b1da01c3d5ab58c39b8f6ea46795457f2022f8f8ebd22fa5ab9"} Jun 02 02:48:22.131754 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:22.131726 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b5b9df7-cjbtx" event={"ID":"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e","Type":"ContainerStarted","Data":"7e4e57b975101ab2214a0b30d0a585f26ed38a6669b29579fa61c00f049ec3f5"} Jun 02 02:48:22.150063 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:22.150016 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-656b5b9df7-cjbtx" podStartSLOduration=1.903342239 podStartE2EDuration="5.150000627s" podCreationTimestamp="2026-06-02 02:48:17 +0000 UTC" firstStartedPulling="2026-06-02 02:48:18.299864849 +0000 UTC m=+178.271465312" lastFinishedPulling="2026-06-02 02:48:21.546523235 +0000 UTC m=+181.518123700" observedRunningTime="2026-06-02 02:48:22.148576658 +0000 UTC m=+182.120177145" watchObservedRunningTime="2026-06-02 02:48:22.150000627 +0000 UTC m=+182.121601113" Jun 02 02:48:23.135547 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:23.135507 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" event={"ID":"a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09","Type":"ContainerStarted","Data":"83eb8746b6337027a1fde70e17d25286a12a6d0c5bcc7a2fe4c4aa88487a6e5b"} Jun 02 02:48:24.138682 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:24.138639 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:24.143837 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:24.143813 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" Jun 02 02:48:24.156848 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:24.156794 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-6brbc" podStartSLOduration=3.765570222 podStartE2EDuration="5.156778756s" podCreationTimestamp="2026-06-02 02:48:19 +0000 UTC" firstStartedPulling="2026-06-02 02:48:21.636765678 +0000 UTC m=+181.608366141" lastFinishedPulling="2026-06-02 02:48:23.027974203 +0000 UTC m=+182.999574675" observedRunningTime="2026-06-02 02:48:24.156531558 +0000 UTC m=+184.128132044" watchObservedRunningTime="2026-06-02 02:48:24.156778756 +0000 UTC m=+184.128379242" Jun 02 02:48:24.937983 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:24.937956 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:28.159068 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:28.159032 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:28.159549 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:28.159091 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:28.164493 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:28.164468 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:29.157467 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:29.157438 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:48:30.418173 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.418123 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-czxf8"] Jun 02 02:48:30.422329 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.422306 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.425018 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.424994 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hs22z\"" Jun 02 02:48:30.425253 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.425235 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 02:48:30.426404 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.426382 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 02:48:30.426404 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.426397 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 02 02:48:30.426560 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.426396 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 02 02:48:30.426560 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.426475 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 02:48:30.432166 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.432110 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-czxf8"] Jun 02 02:48:30.438216 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.438196 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6jdlx"] Jun 02 02:48:30.441290 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.441231 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.441634 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.441461 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8"] Jun 02 02:48:30.444201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.444182 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 02:48:30.444515 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.444500 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 02:48:30.444758 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.444738 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 02:48:30.445171 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.445138 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.445830 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.445810 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mcxww\"" Jun 02 02:48:30.448689 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.448493 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 02 02:48:30.449895 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.449767 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zmszv\"" Jun 02 02:48:30.450091 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.450054 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 02 02:48:30.450216 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.450120 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 02 02:48:30.462718 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.462695 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8"] Jun 02 02:48:30.561493 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561458 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.561670 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d9b95afc-6853-4376-970b-d6412e5a8421-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.561670 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561538 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.561670 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtjmr\" (UniqueName: \"kubernetes.io/projected/d9b95afc-6853-4376-970b-d6412e5a8421-kube-api-access-mtjmr\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.561670 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-sys\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.561879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561672 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.561879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbmnc\" (UniqueName: \"kubernetes.io/projected/a53b89d2-403b-4b13-8c31-48c75074776a-kube-api-access-xbmnc\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.561953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561883 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.561953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.561953 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.562088 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.561969 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562088 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzqch\" (UniqueName: \"kubernetes.io/projected/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-kube-api-access-xzqch\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562187 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562082 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.562187 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562135 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a53b89d2-403b-4b13-8c31-48c75074776a-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.562187 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-metrics-client-ca\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562316 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-root\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562316 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562246 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-wtmp\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562316 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.562437 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.562320 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-textfile\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.663767 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663720 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.663767 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663764 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-textfile\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664018 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663803 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664018 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663834 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d9b95afc-6853-4376-970b-d6412e5a8421-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664018 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663860 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664018 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:30.663894 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 02:48:30.664018 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:30.663971 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls podName:52c03fa6-a0b6-40b7-8fc3-552ae559fbb7 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:31.163948217 +0000 UTC m=+191.135548701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls") pod "node-exporter-6jdlx" (UID: "52c03fa6-a0b6-40b7-8fc3-552ae559fbb7") : secret "node-exporter-tls" not found Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.663896 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mtjmr\" (UniqueName: \"kubernetes.io/projected/d9b95afc-6853-4376-970b-d6412e5a8421-kube-api-access-mtjmr\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664120 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-sys\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664175 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-textfile\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664186 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-sys\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664311 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664253 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xbmnc\" (UniqueName: \"kubernetes.io/projected/a53b89d2-403b-4b13-8c31-48c75074776a-kube-api-access-xbmnc\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.664612 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664311 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.664612 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664345 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664612 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664383 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.664766 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664732 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664826 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664793 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xzqch\" (UniqueName: \"kubernetes.io/projected/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-kube-api-access-xzqch\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.664878 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664831 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.664935 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664885 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a53b89d2-403b-4b13-8c31-48c75074776a-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.664935 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664914 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-metrics-client-ca\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.665034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664934 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d9b95afc-6853-4376-970b-d6412e5a8421-volume-directive-shadow\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.665034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.665001 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-root\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.665034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.664942 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-root\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.665187 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.665074 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-wtmp\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.665269 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.665247 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-wtmp\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.665532 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.665503 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-accelerators-collector-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.666836 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.666079 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.666836 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.666176 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a53b89d2-403b-4b13-8c31-48c75074776a-metrics-client-ca\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.666836 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.666682 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-metrics-client-ca\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.666836 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.666795 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9b95afc-6853-4376-970b-d6412e5a8421-metrics-client-ca\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.667084 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.667050 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.667314 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.667195 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.667402 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.667322 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-tls\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.667402 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.667393 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9b95afc-6853-4376-970b-d6412e5a8421-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.667974 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.667952 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a53b89d2-403b-4b13-8c31-48c75074776a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.680503 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.680441 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtjmr\" (UniqueName: \"kubernetes.io/projected/d9b95afc-6853-4376-970b-d6412e5a8421-kube-api-access-mtjmr\") pod \"kube-state-metrics-8469c486bb-5hsh8\" (UID: \"d9b95afc-6853-4376-970b-d6412e5a8421\") " pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:30.680715 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.680693 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbmnc\" (UniqueName: \"kubernetes.io/projected/a53b89d2-403b-4b13-8c31-48c75074776a-kube-api-access-xbmnc\") pod \"openshift-state-metrics-85658796db-czxf8\" (UID: \"a53b89d2-403b-4b13-8c31-48c75074776a\") " pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.681866 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.681846 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzqch\" (UniqueName: \"kubernetes.io/projected/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-kube-api-access-xzqch\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:30.733937 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.733910 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" Jun 02 02:48:30.759677 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:30.759648 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" Jun 02 02:48:31.170243 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.170202 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:31.170427 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:31.170382 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 02:48:31.170497 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:31.170460 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls podName:52c03fa6-a0b6-40b7-8fc3-552ae559fbb7 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:32.170439962 +0000 UTC m=+192.142040426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls") pod "node-exporter-6jdlx" (UID: "52c03fa6-a0b6-40b7-8fc3-552ae559fbb7") : secret "node-exporter-tls" not found Jun 02 02:48:31.544619 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.544544 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 02:48:31.549445 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.549426 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.552221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552192 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 02 02:48:31.552221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552211 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 02 02:48:31.552557 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552541 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 02 02:48:31.552659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552566 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 02 02:48:31.552659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552620 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 02 02:48:31.552659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552648 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 02 02:48:31.552930 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552906 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 02 02:48:31.553013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552932 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 02 02:48:31.553013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552971 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 02 02:48:31.553013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.552912 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fx92q\"" Jun 02 02:48:31.564333 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.564299 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 02:48:31.673998 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.673963 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-web-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674182 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674182 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674182 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674164 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674209 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674253 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-out\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674336 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674595 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674368 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-volume\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674595 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674396 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674595 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kjrx\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-kube-api-access-6kjrx\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.674595 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.674486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.774962 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.774928 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.775200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.774973 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.775200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.775005 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.775932 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.775991 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-out\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776023 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776045 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776055 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776478 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776514 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-volume\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776546 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776577 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6kjrx\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-kube-api-access-6kjrx\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776644 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776688 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-web-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.776759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.776717 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.779954 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.779324 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.780089 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.780060 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.780632 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:31.780608 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 02 02:48:31.780731 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:31.780691 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls podName:d05b5cb2-31ca-4802-8bd2-ad63ba7c352b nodeName:}" failed. No retries permitted until 2026-06-02 02:48:32.280669575 +0000 UTC m=+192.252270048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d05b5cb2-31ca-4802-8bd2-ad63ba7c352b") : secret "alertmanager-main-tls" not found Jun 02 02:48:31.782684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.782644 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-volume\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.783248 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.783084 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.784465 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.784436 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-config-out\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.784902 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.784876 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-web-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.784998 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.784975 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.785779 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.785642 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.785779 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.785772 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:31.788571 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:31.788519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kjrx\" (UniqueName: \"kubernetes.io/projected/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-kube-api-access-6kjrx\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:32.180960 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.180925 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:32.183512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.183490 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/52c03fa6-a0b6-40b7-8fc3-552ae559fbb7-node-exporter-tls\") pod \"node-exporter-6jdlx\" (UID: \"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7\") " pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:32.253198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.253171 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6jdlx" Jun 02 02:48:32.282084 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.282055 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:32.284640 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.284621 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d05b5cb2-31ca-4802-8bd2-ad63ba7c352b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:32.461299 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:32.461227 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 02 02:48:33.014466 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:33.014438 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c03fa6_a0b6_40b7_8fc3_552ae559fbb7.slice/crio-af00e84f7d0060adaf37381b732fd9e75e7feea25c632676a8f0cac16c4828ac WatchSource:0}: Error finding container af00e84f7d0060adaf37381b732fd9e75e7feea25c632676a8f0cac16c4828ac: Status 404 returned error can't find the container with id af00e84f7d0060adaf37381b732fd9e75e7feea25c632676a8f0cac16c4828ac Jun 02 02:48:33.162742 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.162712 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8"] Jun 02 02:48:33.166246 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.166213 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jdlx" event={"ID":"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7","Type":"ContainerStarted","Data":"af00e84f7d0060adaf37381b732fd9e75e7feea25c632676a8f0cac16c4828ac"} Jun 02 02:48:33.170504 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:33.170452 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b95afc_6853_4376_970b_d6412e5a8421.slice/crio-2cc0dcb71c52305105be4796ddefee9cffb7dc2d782e7fb178c8b4eeb18d2dd0 WatchSource:0}: Error finding container 2cc0dcb71c52305105be4796ddefee9cffb7dc2d782e7fb178c8b4eeb18d2dd0: Status 404 returned error can't find the container with id 2cc0dcb71c52305105be4796ddefee9cffb7dc2d782e7fb178c8b4eeb18d2dd0 Jun 02 02:48:33.177200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.177177 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-85658796db-czxf8"] Jun 02 02:48:33.179048 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:33.179015 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53b89d2_403b_4b13_8c31_48c75074776a.slice/crio-47289eced2e14c8305fce5859a536f0ca2aa30d3ad9fce513b445666a7192622 WatchSource:0}: Error finding container 47289eced2e14c8305fce5859a536f0ca2aa30d3ad9fce513b445666a7192622: Status 404 returned error can't find the container with id 47289eced2e14c8305fce5859a536f0ca2aa30d3ad9fce513b445666a7192622 Jun 02 02:48:33.204741 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.204716 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 02 02:48:33.207847 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:33.207821 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05b5cb2_31ca_4802_8bd2_ad63ba7c352b.slice/crio-c241d6ae48c65bfdb6c2abd858fc1ef8a7051e214a0a600cd149fc419b936971 WatchSource:0}: Error finding container c241d6ae48c65bfdb6c2abd858fc1ef8a7051e214a0a600cd149fc419b936971: Status 404 returned error can't find the container with id c241d6ae48c65bfdb6c2abd858fc1ef8a7051e214a0a600cd149fc419b936971 Jun 02 02:48:33.510202 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.509421 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq"] Jun 02 02:48:33.512863 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.512826 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.516355 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.516049 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 02 02:48:33.516355 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.516190 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 02 02:48:33.517653 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.517626 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 02 02:48:33.519221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.517972 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-60u9061qbeh1m\"" Jun 02 02:48:33.519221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.518423 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 02 02:48:33.519221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.518529 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 02 02:48:33.519221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.518589 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9vn85\"" Jun 02 02:48:33.541568 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.541520 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq"] Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-grpc-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593392 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-metrics-client-ca\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593435 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.593673 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.593486 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzt7\" (UniqueName: \"kubernetes.io/projected/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-kube-api-access-8bzt7\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694483 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694442 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzt7\" (UniqueName: \"kubernetes.io/projected/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-kube-api-access-8bzt7\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694663 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694521 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-grpc-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694663 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694561 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694663 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694594 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694663 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694632 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694912 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694679 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694912 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694702 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-metrics-client-ca\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.694912 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.694739 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.696517 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.696484 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-metrics-client-ca\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.699436 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.699396 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.700217 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.700162 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.700217 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.700185 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-grpc-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.700807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.700753 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.701758 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.701727 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-tls\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.702557 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.702534 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.705560 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.705515 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzt7\" (UniqueName: \"kubernetes.io/projected/91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e-kube-api-access-8bzt7\") pod \"thanos-querier-59c44fb4b9-s5hkq\" (UID: \"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e\") " pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:33.826868 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:33.826404 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:34.176483 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.176449 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" event={"ID":"a53b89d2-403b-4b13-8c31-48c75074776a","Type":"ContainerStarted","Data":"5d45efe721811d30a87a983390e1ae3babebc093b860488fd7b2a56413aefd80"} Jun 02 02:48:34.176825 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.176494 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" event={"ID":"a53b89d2-403b-4b13-8c31-48c75074776a","Type":"ContainerStarted","Data":"6dde17ce6d4a11a3246e37661b3a226d30b07d089eeeb362710fed4537f0bfc2"} Jun 02 02:48:34.176825 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.176508 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" event={"ID":"a53b89d2-403b-4b13-8c31-48c75074776a","Type":"ContainerStarted","Data":"47289eced2e14c8305fce5859a536f0ca2aa30d3ad9fce513b445666a7192622"} Jun 02 02:48:34.180208 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.180138 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"c241d6ae48c65bfdb6c2abd858fc1ef8a7051e214a0a600cd149fc419b936971"} Jun 02 02:48:34.189763 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.186814 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-lx7qk" event={"ID":"dd45154e-530c-4be5-9cca-37172cf641f6","Type":"ContainerStarted","Data":"22fd61152c13e6dd14c7a6eb20224debf9245246bccc24c4e78873304879c201"} Jun 02 02:48:34.189763 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.189574 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:34.193563 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.193513 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" event={"ID":"d9b95afc-6853-4376-970b-d6412e5a8421","Type":"ContainerStarted","Data":"2cc0dcb71c52305105be4796ddefee9cffb7dc2d782e7fb178c8b4eeb18d2dd0"} Jun 02 02:48:34.207927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.207577 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-lx7qk" Jun 02 02:48:34.209257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.209209 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-lx7qk" podStartSLOduration=2.4625694830000002 podStartE2EDuration="20.209193636s" podCreationTimestamp="2026-06-02 02:48:14 +0000 UTC" firstStartedPulling="2026-06-02 02:48:15.373402912 +0000 UTC m=+175.345003375" lastFinishedPulling="2026-06-02 02:48:33.12002705 +0000 UTC m=+193.091627528" observedRunningTime="2026-06-02 02:48:34.207763948 +0000 UTC m=+194.179364463" watchObservedRunningTime="2026-06-02 02:48:34.209193636 +0000 UTC m=+194.180794130" Jun 02 02:48:34.284479 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:34.284416 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq"] Jun 02 02:48:34.300303 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:34.300264 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f27e28_4a03_4b05_bb6d_ecd8c7e6dd4e.slice/crio-23dd8070363f6184a0547d52986a266af72f7c7f1de473575b998a5516b453a9 WatchSource:0}: Error finding container 23dd8070363f6184a0547d52986a266af72f7c7f1de473575b998a5516b453a9: Status 404 returned error can't find the container with id 23dd8070363f6184a0547d52986a266af72f7c7f1de473575b998a5516b453a9 Jun 02 02:48:35.083713 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.083681 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:48:35.198832 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.198783 2563 generic.go:358] "Generic (PLEG): container finished" podID="52c03fa6-a0b6-40b7-8fc3-552ae559fbb7" containerID="d77d237c3679f5314e89bf7708a272f9ee359ac242ae4671e03ce5b65017c9bd" exitCode=0 Jun 02 02:48:35.199299 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.198885 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jdlx" event={"ID":"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7","Type":"ContainerDied","Data":"d77d237c3679f5314e89bf7708a272f9ee359ac242ae4671e03ce5b65017c9bd"} Jun 02 02:48:35.200596 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.200565 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"23dd8070363f6184a0547d52986a266af72f7c7f1de473575b998a5516b453a9"} Jun 02 02:48:35.213706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.213652 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk"] Jun 02 02:48:35.228832 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.228808 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:35.229004 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.228977 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk"] Jun 02 02:48:35.231583 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.231544 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-dwqjk\"" Jun 02 02:48:35.231758 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.231739 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 02:48:35.314620 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.314553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-npcbk\" (UID: \"b260d2de-4531-4fda-a124-cb4ea40525e1\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:35.421234 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.421198 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-npcbk\" (UID: \"b260d2de-4531-4fda-a124-cb4ea40525e1\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:35.421428 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:35.421409 2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 02:48:35.421498 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:48:35.421484 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert podName:b260d2de-4531-4fda-a124-cb4ea40525e1 nodeName:}" failed. No retries permitted until 2026-06-02 02:48:35.92146125 +0000 UTC m=+195.893061713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-npcbk" (UID: "b260d2de-4531-4fda-a124-cb4ea40525e1") : secret "monitoring-plugin-cert" not found Jun 02 02:48:35.829718 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.829683 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:48:35.849290 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.849264 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:48:35.849447 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.849384 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:35.858569 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.858370 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 02:48:35.927380 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.927345 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-npcbk\" (UID: \"b260d2de-4531-4fda-a124-cb4ea40525e1\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:35.932173 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:35.932105 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b260d2de-4531-4fda-a124-cb4ea40525e1-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-npcbk\" (UID: \"b260d2de-4531-4fda-a124-cb4ea40525e1\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028416 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028453 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028545 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n8zj\" (UniqueName: \"kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.028807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.028617 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130553 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130493 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130553 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130555 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5n8zj\" (UniqueName: \"kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130634 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130701 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130735 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130775 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.130816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.130805 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.132257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.131582 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.132257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.131676 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.132257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.131767 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.132807 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.132765 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.134200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.134137 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.134437 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.134410 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.139267 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.139226 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n8zj\" (UniqueName: \"kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj\") pod \"console-64f7bf66dd-86gzb\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.141901 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.141880 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:36.163956 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.163922 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:36.545996 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.545970 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk"] Jun 02 02:48:36.577513 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.577471 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:48:36.592435 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:36.592389 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc983cec4_49f5_4a72_a3e4_c724b2054f4c.slice/crio-bbb6d69e49b8c0cd54b967daa2012425c0db36ae2302e1dc9b12485c49f54d95 WatchSource:0}: Error finding container bbb6d69e49b8c0cd54b967daa2012425c0db36ae2302e1dc9b12485c49f54d95: Status 404 returned error can't find the container with id bbb6d69e49b8c0cd54b967daa2012425c0db36ae2302e1dc9b12485c49f54d95 Jun 02 02:48:36.739811 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.739712 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 02:48:36.756232 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.754914 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.759427 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.759372 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 02 02:48:36.759427 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.759411 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 02:48:36.760110 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.760088 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ajp160fokbvfj\"" Jun 02 02:48:36.765702 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.765679 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 02 02:48:36.765918 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.765898 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.766132 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qb9hf\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.766337 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.766531 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.766709 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.767316 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 02 02:48:36.767709 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.767518 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 02 02:48:36.771029 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.770603 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 02 02:48:36.771029 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.770920 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 02 02:48:36.778383 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.778335 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 02 02:48:36.785015 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.784801 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 02 02:48:36.793065 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.793033 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 02:48:36.939600 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.939773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.939773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939662 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.939773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.939773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.939773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939795 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.939878 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940204 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940029 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntnw8\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-kube-api-access-ntnw8\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940204 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940204 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940204 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940193 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940286 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940313 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:36.940399 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:36.940341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041610 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041659 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041695 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041723 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041758 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041797 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041835 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041865 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041898 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041928 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.041951 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042007 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042037 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ntnw8\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-kube-api-access-ntnw8\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042094 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042119 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042190 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.043170 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042222 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.044058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.042252 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.044058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.043653 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.046172 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.045470 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.046172 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.045773 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.047746 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.047562 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.049704 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.049654 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.051901 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.051710 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.051901 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.051823 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.052641 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.052534 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.053021 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.052996 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef09f87c-2e5e-4d78-9af3-d67df134c489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.053598 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.053471 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.053598 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.053473 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef09f87c-2e5e-4d78-9af3-d67df134c489-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.053831 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.053807 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.053967 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.053945 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.054021 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.053978 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.054780 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.054756 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.055231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.055207 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.056057 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.056015 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef09f87c-2e5e-4d78-9af3-d67df134c489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.058666 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.058625 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntnw8\" (UniqueName: \"kubernetes.io/projected/ef09f87c-2e5e-4d78-9af3-d67df134c489-kube-api-access-ntnw8\") pod \"prometheus-k8s-0\" (UID: \"ef09f87c-2e5e-4d78-9af3-d67df134c489\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.081575 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.081369 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:48:37.208700 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.207887 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7bf66dd-86gzb" event={"ID":"c983cec4-49f5-4a72-a3e4-c724b2054f4c","Type":"ContainerStarted","Data":"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7"} Jun 02 02:48:37.208700 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.207936 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7bf66dd-86gzb" event={"ID":"c983cec4-49f5-4a72-a3e4-c724b2054f4c","Type":"ContainerStarted","Data":"bbb6d69e49b8c0cd54b967daa2012425c0db36ae2302e1dc9b12485c49f54d95"} Jun 02 02:48:37.209690 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.209651 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" event={"ID":"b260d2de-4531-4fda-a124-cb4ea40525e1","Type":"ContainerStarted","Data":"a2f86353c1e801e6b3b9d72e4d0a5e13ce82ad67d521a8cd88a80aea6492e5fa"} Jun 02 02:48:37.211525 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.211406 2563 generic.go:358] "Generic (PLEG): container finished" podID="d05b5cb2-31ca-4802-8bd2-ad63ba7c352b" containerID="fb2ea552e48fb8b7942fcb08652074cd598457c1136b1cae1452a1f17716b05e" exitCode=0 Jun 02 02:48:37.211525 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.211495 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerDied","Data":"fb2ea552e48fb8b7942fcb08652074cd598457c1136b1cae1452a1f17716b05e"} Jun 02 02:48:37.214766 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.214744 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jdlx" event={"ID":"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7","Type":"ContainerStarted","Data":"8b518b4586a17627528cdf3a1f8b812fcd654c5d4a9c4ddf0e44d39768a9741a"} Jun 02 02:48:37.214835 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.214775 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6jdlx" event={"ID":"52c03fa6-a0b6-40b7-8fc3-552ae559fbb7","Type":"ContainerStarted","Data":"b1b4d328d7a55128c331fc1328aacb9acf983de118e8b6f382a58083329b6280"} Jun 02 02:48:37.217403 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.217377 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" event={"ID":"d9b95afc-6853-4376-970b-d6412e5a8421","Type":"ContainerStarted","Data":"152768f79d163e668f6f4932b77b70f779846a28f5172624a9df5e611542ab14"} Jun 02 02:48:37.217498 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.217409 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" event={"ID":"d9b95afc-6853-4376-970b-d6412e5a8421","Type":"ContainerStarted","Data":"0f9468d2f13f76293a2f56cb688d2ae01a05325335eeb13e2d63a698dac1f901"} Jun 02 02:48:37.217498 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.217423 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" event={"ID":"d9b95afc-6853-4376-970b-d6412e5a8421","Type":"ContainerStarted","Data":"ac70f3f6c64955ca0ec5e6012f14a92d330d7f5e8165afd6bff37be94d7d4235"} Jun 02 02:48:37.219644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.219619 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" event={"ID":"a53b89d2-403b-4b13-8c31-48c75074776a","Type":"ContainerStarted","Data":"a0195d4ad0ded8dc2a6111a677b2c9bf343e2b95a5aec95ffd921da136823963"} Jun 02 02:48:37.229761 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.229607 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-64f7bf66dd-86gzb" podStartSLOduration=2.229592931 podStartE2EDuration="2.229592931s" podCreationTimestamp="2026-06-02 02:48:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:48:37.227392445 +0000 UTC m=+197.198992930" watchObservedRunningTime="2026-06-02 02:48:37.229592931 +0000 UTC m=+197.201193420" Jun 02 02:48:37.275674 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.275589 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6jdlx" podStartSLOduration=6.141460829 podStartE2EDuration="7.275572072s" podCreationTimestamp="2026-06-02 02:48:30 +0000 UTC" firstStartedPulling="2026-06-02 02:48:33.017083392 +0000 UTC m=+192.988683861" lastFinishedPulling="2026-06-02 02:48:34.151194638 +0000 UTC m=+194.122795104" observedRunningTime="2026-06-02 02:48:37.246818232 +0000 UTC m=+197.218418721" watchObservedRunningTime="2026-06-02 02:48:37.275572072 +0000 UTC m=+197.247172558" Jun 02 02:48:37.290755 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.290701 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-85658796db-czxf8" podStartSLOduration=4.263441744 podStartE2EDuration="7.290685987s" podCreationTimestamp="2026-06-02 02:48:30 +0000 UTC" firstStartedPulling="2026-06-02 02:48:33.284229425 +0000 UTC m=+193.255829887" lastFinishedPulling="2026-06-02 02:48:36.31147365 +0000 UTC m=+196.283074130" observedRunningTime="2026-06-02 02:48:37.290208134 +0000 UTC m=+197.261808621" watchObservedRunningTime="2026-06-02 02:48:37.290685987 +0000 UTC m=+197.262286490" Jun 02 02:48:37.310040 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.309987 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-8469c486bb-5hsh8" podStartSLOduration=4.169419296 podStartE2EDuration="7.309970503s" podCreationTimestamp="2026-06-02 02:48:30 +0000 UTC" firstStartedPulling="2026-06-02 02:48:33.172864489 +0000 UTC m=+193.144464963" lastFinishedPulling="2026-06-02 02:48:36.313415697 +0000 UTC m=+196.285016170" observedRunningTime="2026-06-02 02:48:37.308502463 +0000 UTC m=+197.280102949" watchObservedRunningTime="2026-06-02 02:48:37.309970503 +0000 UTC m=+197.281570991" Jun 02 02:48:37.747412 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:37.747380 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 02 02:48:37.752542 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:48:37.752516 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef09f87c_2e5e_4d78_9af3_d67df134c489.slice/crio-a6c548741401bb1470f243b0db0b063c7d88b3e43e44c21c1669f07041863c62 WatchSource:0}: Error finding container a6c548741401bb1470f243b0db0b063c7d88b3e43e44c21c1669f07041863c62: Status 404 returned error can't find the container with id a6c548741401bb1470f243b0db0b063c7d88b3e43e44c21c1669f07041863c62 Jun 02 02:48:38.228102 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:38.228023 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"a4677055b97afdf81704c9df70a11f5269a8c759031504b12005e2f77c5a8747"} Jun 02 02:48:38.228102 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:38.228077 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"d679eb809cc62e6ae548ab2622155fbd8589c9c6eba15a37664b3dc997dae0ea"} Jun 02 02:48:38.230865 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:38.230831 2563 generic.go:358] "Generic (PLEG): container finished" podID="ef09f87c-2e5e-4d78-9af3-d67df134c489" containerID="3bb0b2df094323a965462ef161dcd91f1d955b5ea133d47047f75f817bedb5d2" exitCode=0 Jun 02 02:48:38.231461 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:38.231319 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerDied","Data":"3bb0b2df094323a965462ef161dcd91f1d955b5ea133d47047f75f817bedb5d2"} Jun 02 02:48:38.231461 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:38.231353 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"a6c548741401bb1470f243b0db0b063c7d88b3e43e44c21c1669f07041863c62"} Jun 02 02:48:39.239044 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:39.239008 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"b2bb0210fff84de32c97672fcb32c32184be449ba0630d43d3fbb4bb6c22fbe7"} Jun 02 02:48:39.954450 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:39.953990 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" podUID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" containerName="registry" containerID="cri-o://9449feff78979d7a9c3858326e0d967cfdb966b84987745b65a83ebabeda249b" gracePeriod=30 Jun 02 02:48:40.244924 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.244775 2563 generic.go:358] "Generic (PLEG): container finished" podID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" containerID="9449feff78979d7a9c3858326e0d967cfdb966b84987745b65a83ebabeda249b" exitCode=0 Jun 02 02:48:40.244924 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.244874 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" event={"ID":"a91b6a74-7d69-4950-9ada-a6d91ac2ea22","Type":"ContainerDied","Data":"9449feff78979d7a9c3858326e0d967cfdb966b84987745b65a83ebabeda249b"} Jun 02 02:48:40.284817 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.284792 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:40.378226 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378194 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378245 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378268 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378308 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378362 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378393 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378457 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378431 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8glzc\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.378728 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.378470 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates\") pod \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\" (UID: \"a91b6a74-7d69-4950-9ada-a6d91ac2ea22\") " Jun 02 02:48:40.379482 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.379202 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:48:40.380014 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.379972 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:48:40.389448 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.389108 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:48:40.389448 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.389191 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:48:40.389448 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.389263 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc" (OuterVolumeSpecName: "kube-api-access-8glzc") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "kube-api-access-8glzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:48:40.389448 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.389381 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:48:40.390666 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.390622 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:48:40.390738 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.390690 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a91b6a74-7d69-4950-9ada-a6d91ac2ea22" (UID: "a91b6a74-7d69-4950-9ada-a6d91ac2ea22"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:48:40.479454 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479378 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-image-registry-private-configuration\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479454 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479414 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-ca-trust-extracted\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479454 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479429 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8glzc\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-kube-api-access-8glzc\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479454 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479444 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-certificates\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479747 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479458 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-installation-pull-secrets\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479747 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479473 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-registry-tls\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479747 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479486 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-trusted-ca\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.479747 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.479513 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a91b6a74-7d69-4950-9ada-a6d91ac2ea22-bound-sa-token\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:48:40.905257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:40.904716 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:48:41.250167 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.250052 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" event={"ID":"b260d2de-4531-4fda-a124-cb4ea40525e1","Type":"ContainerStarted","Data":"e56dd7c8220ed94cfe1902c066b67a2b6854d34d0527536ae93e516ddc1c98ab"} Jun 02 02:48:41.250639 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.250509 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:41.252407 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.252383 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" event={"ID":"a91b6a74-7d69-4950-9ada-a6d91ac2ea22","Type":"ContainerDied","Data":"46f6b4b8750acea702ca0d8b4ec6ebbabdd91dbf99b2d2fba9d6b38199937e33"} Jun 02 02:48:41.252539 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.252418 2563 scope.go:117] "RemoveContainer" containerID="9449feff78979d7a9c3858326e0d967cfdb966b84987745b65a83ebabeda249b" Jun 02 02:48:41.252539 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.252514 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666d6d74cc-zltsz" Jun 02 02:48:41.256610 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.256589 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"e76737792560c3bd397e87f7460c95fb661bef2b90dc99cfe5e75591772572e6"} Jun 02 02:48:41.258590 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.258572 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" Jun 02 02:48:41.276075 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.276012 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-npcbk" podStartSLOduration=2.7511781969999998 podStartE2EDuration="6.275995581s" podCreationTimestamp="2026-06-02 02:48:35 +0000 UTC" firstStartedPulling="2026-06-02 02:48:36.601767444 +0000 UTC m=+196.573367912" lastFinishedPulling="2026-06-02 02:48:40.126584817 +0000 UTC m=+200.098185296" observedRunningTime="2026-06-02 02:48:41.275248342 +0000 UTC m=+201.246848828" watchObservedRunningTime="2026-06-02 02:48:41.275995581 +0000 UTC m=+201.247596066" Jun 02 02:48:41.310421 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.310393 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:48:41.313871 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:41.313848 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-666d6d74cc-zltsz"] Jun 02 02:48:42.262886 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:42.262845 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"3fa89490df4444f4d2edc1309557cb3990d27e3f30beb0000bd56b0c89513773"} Jun 02 02:48:42.265138 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:42.265103 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"4912e6244a2c81ebdabf5af4ebca47682fb048b7d1075f27b9249153cbf1fc9a"} Jun 02 02:48:42.509858 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:42.509829 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" path="/var/lib/kubelet/pods/a91b6a74-7d69-4950-9ada-a6d91ac2ea22/volumes" Jun 02 02:48:43.271664 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.271636 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"3ec3ef0125dab74305b3b4349208599c1921573326464a9d70da9c834ef72d06"} Jun 02 02:48:43.272022 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.271676 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"6d8a0b420da317d3df5cd35e657b82575660170f88765d7d88170408ca75a202"} Jun 02 02:48:43.272022 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.271690 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"4a2451dcf964feafed28b8b15fd1b41384e5df2203c7826ee3f9ad0a931fe374"} Jun 02 02:48:43.272022 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.271705 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d05b5cb2-31ca-4802-8bd2-ad63ba7c352b","Type":"ContainerStarted","Data":"e3ab898f1f673236ccfc09d015aba732a93cf54db8407e3d9581f0c92dca03ab"} Jun 02 02:48:43.274355 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.274331 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"7a65ef843fd7544f4b4f4897bf330085fac4ef5cd039c7c5a9f7390e56c2ec27"} Jun 02 02:48:43.274475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.274364 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" event={"ID":"91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e","Type":"ContainerStarted","Data":"d681bf76616b98acea02ae5415bb0405b39625026e6c563d326b602674604330"} Jun 02 02:48:43.274589 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.274538 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:43.276691 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.276666 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"97967b90db7a76be46d028915bcbee3327b9262a74391e4d4a5e2d4f14ef3987"} Jun 02 02:48:43.276809 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.276706 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"269e75ba6e90e2ce7e0f24fbd30601902b4266c8599fcfeadb852d7876236857"} Jun 02 02:48:43.276809 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.276721 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"e3b0ca278b1a730ea26e41cc6554dac9423e1acce7b7cd1e668f9d67e6af0824"} Jun 02 02:48:43.280998 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.280979 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" Jun 02 02:48:43.302682 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.302638 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.62608118 podStartE2EDuration="12.302626497s" podCreationTimestamp="2026-06-02 02:48:31 +0000 UTC" firstStartedPulling="2026-06-02 02:48:33.209898408 +0000 UTC m=+193.181498871" lastFinishedPulling="2026-06-02 02:48:40.886443709 +0000 UTC m=+200.858044188" observedRunningTime="2026-06-02 02:48:43.300474692 +0000 UTC m=+203.272075193" watchObservedRunningTime="2026-06-02 02:48:43.302626497 +0000 UTC m=+203.274226979" Jun 02 02:48:43.323107 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:43.323065 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59c44fb4b9-s5hkq" podStartSLOduration=3.312432493 podStartE2EDuration="10.323051002s" podCreationTimestamp="2026-06-02 02:48:33 +0000 UTC" firstStartedPulling="2026-06-02 02:48:34.308330104 +0000 UTC m=+194.279930570" lastFinishedPulling="2026-06-02 02:48:41.318948603 +0000 UTC m=+201.290549079" observedRunningTime="2026-06-02 02:48:43.321350763 +0000 UTC m=+203.292951275" watchObservedRunningTime="2026-06-02 02:48:43.323051002 +0000 UTC m=+203.294651486" Jun 02 02:48:44.282467 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:44.282432 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"183b3903e6cf8ef1d24e9846ed33960b2eef4fa034110d28c5368b335a7cb617"} Jun 02 02:48:44.282467 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:44.282472 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"e80e0c316e3e8666ab93a5f1d792d64d971b2ffcf731bf0efc44cd1b29de1fb6"} Jun 02 02:48:44.283056 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:44.282486 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef09f87c-2e5e-4d78-9af3-d67df134c489","Type":"ContainerStarted","Data":"b61085164b0521d4d79364d03cc1c28d5a8d9b1285cbc6d7771a6a41730f9e94"} Jun 02 02:48:44.312511 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:44.312462 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.713858863 podStartE2EDuration="8.312448785s" podCreationTimestamp="2026-06-02 02:48:36 +0000 UTC" firstStartedPulling="2026-06-02 02:48:38.232816933 +0000 UTC m=+198.204417410" lastFinishedPulling="2026-06-02 02:48:42.831406866 +0000 UTC m=+202.803007332" observedRunningTime="2026-06-02 02:48:44.310353157 +0000 UTC m=+204.281953642" watchObservedRunningTime="2026-06-02 02:48:44.312448785 +0000 UTC m=+204.284049269" Jun 02 02:48:46.164277 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:46.164244 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:48:47.082308 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:48:47.082276 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:49:00.110986 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.110930 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-656b5b9df7-cjbtx" podUID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" containerName="console" containerID="cri-o://7e4e57b975101ab2214a0b30d0a585f26ed38a6669b29579fa61c00f049ec3f5" gracePeriod=15 Jun 02 02:49:00.333449 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.333424 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-656b5b9df7-cjbtx_6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e/console/0.log" Jun 02 02:49:00.333553 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.333467 2563 generic.go:358] "Generic (PLEG): container finished" podID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" containerID="7e4e57b975101ab2214a0b30d0a585f26ed38a6669b29579fa61c00f049ec3f5" exitCode=2 Jun 02 02:49:00.333553 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.333540 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b5b9df7-cjbtx" event={"ID":"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e","Type":"ContainerDied","Data":"7e4e57b975101ab2214a0b30d0a585f26ed38a6669b29579fa61c00f049ec3f5"} Jun 02 02:49:00.333626 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.333571 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-656b5b9df7-cjbtx" event={"ID":"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e","Type":"ContainerDied","Data":"73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d"} Jun 02 02:49:00.333626 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.333581 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73015302ad9c605417bb17b137a4f52c738126f2e495795004843c857d07170d" Jun 02 02:49:00.341179 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.341162 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-656b5b9df7-cjbtx_6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e/console/0.log" Jun 02 02:49:00.341268 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.341215 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:49:00.455630 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455601 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.455796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455641 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.455796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455665 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.455796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455705 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbsbm\" (UniqueName: \"kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.455796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455729 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.455796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.455748 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config\") pod \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\" (UID: \"6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e\") " Jun 02 02:49:00.456113 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.456085 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca" (OuterVolumeSpecName: "service-ca") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:00.456213 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.456095 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:00.456213 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.456193 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config" (OuterVolumeSpecName: "console-config") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:00.458042 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.458011 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:49:00.458120 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.458023 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm" (OuterVolumeSpecName: "kube-api-access-rbsbm") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "kube-api-access-rbsbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:49:00.458120 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.458074 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" (UID: "6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:49:00.556466 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556438 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-oauth-serving-cert\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:00.556466 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556463 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-config\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:00.556609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556474 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-service-ca\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:00.556609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556482 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbsbm\" (UniqueName: \"kubernetes.io/projected/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-kube-api-access-rbsbm\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:00.556609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556491 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-serving-cert\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:00.556609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:00.556500 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e-console-oauth-config\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:01.336347 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:01.336314 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-656b5b9df7-cjbtx" Jun 02 02:49:01.352157 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:01.352124 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:49:01.356180 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:01.356159 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-656b5b9df7-cjbtx"] Jun 02 02:49:02.510286 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:02.510255 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" path="/var/lib/kubelet/pods/6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e/volumes" Jun 02 02:49:05.934115 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:05.934055 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-64f7bf66dd-86gzb" podUID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" containerName="console" containerID="cri-o://b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7" gracePeriod=15 Jun 02 02:49:06.216875 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.216849 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64f7bf66dd-86gzb_c983cec4-49f5-4a72-a3e4-c724b2054f4c/console/0.log" Jun 02 02:49:06.217016 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.216928 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:49:06.309206 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309172 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309206 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309215 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309452 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309239 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5n8zj\" (UniqueName: \"kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309452 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309293 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309452 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309308 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309603 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309487 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309603 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309547 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert\") pod \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\" (UID: \"c983cec4-49f5-4a72-a3e4-c724b2054f4c\") " Jun 02 02:49:06.309707 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309628 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca" (OuterVolumeSpecName: "service-ca") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:06.309707 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309672 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:06.309791 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309749 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:06.309791 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309744 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config" (OuterVolumeSpecName: "console-config") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:49:06.309874 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309859 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-config\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.309938 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309877 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-oauth-serving-cert\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.309938 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309887 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-service-ca\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.309938 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.309896 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c983cec4-49f5-4a72-a3e4-c724b2054f4c-trusted-ca-bundle\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.311554 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.311522 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:49:06.311650 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.311618 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj" (OuterVolumeSpecName: "kube-api-access-5n8zj") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "kube-api-access-5n8zj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:49:06.311650 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.311617 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c983cec4-49f5-4a72-a3e4-c724b2054f4c" (UID: "c983cec4-49f5-4a72-a3e4-c724b2054f4c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 02:49:06.352791 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352768 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64f7bf66dd-86gzb_c983cec4-49f5-4a72-a3e4-c724b2054f4c/console/0.log" Jun 02 02:49:06.352922 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352804 2563 generic.go:358] "Generic (PLEG): container finished" podID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" containerID="b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7" exitCode=2 Jun 02 02:49:06.352922 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352836 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7bf66dd-86gzb" event={"ID":"c983cec4-49f5-4a72-a3e4-c724b2054f4c","Type":"ContainerDied","Data":"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7"} Jun 02 02:49:06.352922 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352871 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f7bf66dd-86gzb" Jun 02 02:49:06.352922 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352886 2563 scope.go:117] "RemoveContainer" containerID="b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7" Jun 02 02:49:06.353105 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.352875 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f7bf66dd-86gzb" event={"ID":"c983cec4-49f5-4a72-a3e4-c724b2054f4c","Type":"ContainerDied","Data":"bbb6d69e49b8c0cd54b967daa2012425c0db36ae2302e1dc9b12485c49f54d95"} Jun 02 02:49:06.361882 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.361847 2563 scope.go:117] "RemoveContainer" containerID="b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7" Jun 02 02:49:06.362175 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:49:06.362130 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7\": container with ID starting with b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7 not found: ID does not exist" containerID="b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7" Jun 02 02:49:06.362261 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.362180 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7"} err="failed to get container status \"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7\": rpc error: code = NotFound desc = could not find container \"b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7\": container with ID starting with b7c5a8e38cbb0a625becadb0b6b04f5ba9b94b26a74c63d814b25b5da8d041a7 not found: ID does not exist" Jun 02 02:49:06.378283 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.378253 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:49:06.379397 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.379378 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f7bf66dd-86gzb"] Jun 02 02:49:06.410621 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.410587 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-oauth-config\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.410621 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.410615 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c983cec4-49f5-4a72-a3e4-c724b2054f4c-console-serving-cert\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.410621 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.410628 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5n8zj\" (UniqueName: \"kubernetes.io/projected/c983cec4-49f5-4a72-a3e4-c724b2054f4c-kube-api-access-5n8zj\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:49:06.510848 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:06.510772 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" path="/var/lib/kubelet/pods/c983cec4-49f5-4a72-a3e4-c724b2054f4c/volumes" Jun 02 02:49:20.393981 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.393897 2563 generic.go:358] "Generic (PLEG): container finished" podID="6f60493a-6291-4ded-971b-8c016d0fc23e" containerID="f18a8b9426304da29a2bddc45af7782bdcdad3e9db781dd7b06b74c870295d4c" exitCode=0 Jun 02 02:49:20.394475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.393976 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" event={"ID":"6f60493a-6291-4ded-971b-8c016d0fc23e","Type":"ContainerDied","Data":"f18a8b9426304da29a2bddc45af7782bdcdad3e9db781dd7b06b74c870295d4c"} Jun 02 02:49:20.394475 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.394349 2563 scope.go:117] "RemoveContainer" containerID="f18a8b9426304da29a2bddc45af7782bdcdad3e9db781dd7b06b74c870295d4c" Jun 02 02:49:20.395330 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.395303 2563 generic.go:358] "Generic (PLEG): container finished" podID="ec240069-31bf-4333-b5c0-90829b8c73f0" containerID="b9eab5e68cc8a6c0dc72a5913d03dda47840830c5554bedd09f1b1c9218ca99d" exitCode=0 Jun 02 02:49:20.395426 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.395352 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" event={"ID":"ec240069-31bf-4333-b5c0-90829b8c73f0","Type":"ContainerDied","Data":"b9eab5e68cc8a6c0dc72a5913d03dda47840830c5554bedd09f1b1c9218ca99d"} Jun 02 02:49:20.395688 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:20.395665 2563 scope.go:117] "RemoveContainer" containerID="b9eab5e68cc8a6c0dc72a5913d03dda47840830c5554bedd09f1b1c9218ca99d" Jun 02 02:49:21.399772 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:21.399735 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2rd92" event={"ID":"6f60493a-6291-4ded-971b-8c016d0fc23e","Type":"ContainerStarted","Data":"085d37d875315730653c9df59f59ffcc50068dac01aa4b4f1a0b1717d6c37108"} Jun 02 02:49:21.401513 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:21.401490 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-788bf58df5-dt6tw" event={"ID":"ec240069-31bf-4333-b5c0-90829b8c73f0","Type":"ContainerStarted","Data":"66d0e8037545fe7f7052b99075fb274c899af715704f20db96074123a4a402de"} Jun 02 02:49:32.322335 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:32.322298 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:49:32.324447 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:32.324427 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d6cad5-7d21-4aaf-b31a-9d1995494fd0-metrics-certs\") pod \"network-metrics-daemon-rml5c\" (UID: \"70d6cad5-7d21-4aaf-b31a-9d1995494fd0\") " pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:49:32.610597 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:32.610527 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgk2p\"" Jun 02 02:49:32.618033 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:32.618011 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rml5c" Jun 02 02:49:32.769191 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:32.769169 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rml5c"] Jun 02 02:49:32.771969 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:49:32.771935 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70d6cad5_7d21_4aaf_b31a_9d1995494fd0.slice/crio-f7ebd4e8094137f4e46612f00930cbf19a708fb2f02400366c8a7b54fddebae7 WatchSource:0}: Error finding container f7ebd4e8094137f4e46612f00930cbf19a708fb2f02400366c8a7b54fddebae7: Status 404 returned error can't find the container with id f7ebd4e8094137f4e46612f00930cbf19a708fb2f02400366c8a7b54fddebae7 Jun 02 02:49:33.441945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:33.441899 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rml5c" event={"ID":"70d6cad5-7d21-4aaf-b31a-9d1995494fd0","Type":"ContainerStarted","Data":"f7ebd4e8094137f4e46612f00930cbf19a708fb2f02400366c8a7b54fddebae7"} Jun 02 02:49:34.446463 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:34.446422 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rml5c" event={"ID":"70d6cad5-7d21-4aaf-b31a-9d1995494fd0","Type":"ContainerStarted","Data":"734d6538f0d336c54bad6d0b238de00f291e4b6f514ecb9529f55223e415fbb8"} Jun 02 02:49:34.446463 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:34.446463 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rml5c" event={"ID":"70d6cad5-7d21-4aaf-b31a-9d1995494fd0","Type":"ContainerStarted","Data":"85ec12c56fe02d20752c813475950178aa22bfcccea2e48d5da67671b7aeb93d"} Jun 02 02:49:34.463346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:34.463298 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rml5c" podStartSLOduration=253.333848884 podStartE2EDuration="4m14.463283917s" podCreationTimestamp="2026-06-02 02:45:20 +0000 UTC" firstStartedPulling="2026-06-02 02:49:32.774078779 +0000 UTC m=+252.745679242" lastFinishedPulling="2026-06-02 02:49:33.903513796 +0000 UTC m=+253.875114275" observedRunningTime="2026-06-02 02:49:34.462120596 +0000 UTC m=+254.433721084" watchObservedRunningTime="2026-06-02 02:49:34.463283917 +0000 UTC m=+254.434884402" Jun 02 02:49:37.081978 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:37.081943 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:49:37.101816 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:37.101791 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:49:37.473545 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:49:37.473517 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 02 02:50:20.398819 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:50:20.398788 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:50:20.399338 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:50:20.398965 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:50:20.401658 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:50:20.401636 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:50:20.401830 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:50:20.401810 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:50:20.407851 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:50:20.407834 2563 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 02:51:57.568111 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568074 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d"] Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568401 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" containerName="registry" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568412 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" containerName="registry" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568429 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568434 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568442 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568448 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568519 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="c983cec4-49f5-4a72-a3e4-c724b2054f4c" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568529 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b0feceb-2fc2-4dcf-9a0c-707bdf645a4e" containerName="console" Jun 02 02:51:57.568684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.568535 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="a91b6a74-7d69-4950-9ada-a6d91ac2ea22" containerName="registry" Jun 02 02:51:57.571024 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.571001 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.573643 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.573625 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 02:51:57.574650 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.574633 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hqqxq\"" Jun 02 02:51:57.574650 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.574644 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 02:51:57.580991 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.580966 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d"] Jun 02 02:51:57.655071 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.655039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbaabd11-9a12-496b-921b-199d13c576c9-tmp\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.655304 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.655081 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnp8x\" (UniqueName: \"kubernetes.io/projected/fbaabd11-9a12-496b-921b-199d13c576c9-kube-api-access-jnp8x\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.756097 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.756064 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbaabd11-9a12-496b-921b-199d13c576c9-tmp\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.756266 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.756105 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jnp8x\" (UniqueName: \"kubernetes.io/projected/fbaabd11-9a12-496b-921b-199d13c576c9-kube-api-access-jnp8x\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.756444 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.756426 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbaabd11-9a12-496b-921b-199d13c576c9-tmp\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.764548 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.764526 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnp8x\" (UniqueName: \"kubernetes.io/projected/fbaabd11-9a12-496b-921b-199d13c576c9-kube-api-access-jnp8x\") pod \"openshift-lws-operator-bfc7f696d-55h4d\" (UID: \"fbaabd11-9a12-496b-921b-199d13c576c9\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:57.892385 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:57.892362 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" Jun 02 02:51:58.015447 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:58.015422 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d"] Jun 02 02:51:58.017835 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:51:58.017807 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbaabd11_9a12_496b_921b_199d13c576c9.slice/crio-58dec2fb4810f5e13c3312b51f78cadc62d8477eefbc06ac3323e084541dda77 WatchSource:0}: Error finding container 58dec2fb4810f5e13c3312b51f78cadc62d8477eefbc06ac3323e084541dda77: Status 404 returned error can't find the container with id 58dec2fb4810f5e13c3312b51f78cadc62d8477eefbc06ac3323e084541dda77 Jun 02 02:51:58.019270 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:58.019255 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 02:51:58.873778 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:51:58.873742 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" event={"ID":"fbaabd11-9a12-496b-921b-199d13c576c9","Type":"ContainerStarted","Data":"58dec2fb4810f5e13c3312b51f78cadc62d8477eefbc06ac3323e084541dda77"} Jun 02 02:52:01.884737 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:01.884697 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" event={"ID":"fbaabd11-9a12-496b-921b-199d13c576c9","Type":"ContainerStarted","Data":"ed85bc7c40066a5a46e5a114063290d7d6f07937dc8cee36770b56d8807606eb"} Jun 02 02:52:01.900966 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:01.900915 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-55h4d" podStartSLOduration=2.038870596 podStartE2EDuration="4.900902973s" podCreationTimestamp="2026-06-02 02:51:57 +0000 UTC" firstStartedPulling="2026-06-02 02:51:58.019379658 +0000 UTC m=+397.990980121" lastFinishedPulling="2026-06-02 02:52:00.881412035 +0000 UTC m=+400.853012498" observedRunningTime="2026-06-02 02:52:01.898569113 +0000 UTC m=+401.870169599" watchObservedRunningTime="2026-06-02 02:52:01.900902973 +0000 UTC m=+401.872503461" Jun 02 02:52:17.468394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.468361 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw"] Jun 02 02:52:17.471773 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.471753 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.476214 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.476187 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 02:52:17.476350 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.476221 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 02:52:17.476597 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.476582 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4m85r\"" Jun 02 02:52:17.476982 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.476961 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 02:52:17.477240 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.477221 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 02:52:17.501637 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.501617 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw"] Jun 02 02:52:17.615655 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.615628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.615805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.615666 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.615805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.615692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg5rx\" (UniqueName: \"kubernetes.io/projected/c3b48e53-1f3c-4143-8b10-766db3ea33fc-kube-api-access-qg5rx\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.615805 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.615791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.615917 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.615826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717064 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717029 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717084 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717135 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717317 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717194 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717317 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717232 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qg5rx\" (UniqueName: \"kubernetes.io/projected/c3b48e53-1f3c-4143-8b10-766db3ea33fc-kube-api-access-qg5rx\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.717549 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.717524 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-manifests\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.719217 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.719132 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b48e53-1f3c-4143-8b10-766db3ea33fc-tmp\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.719472 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.719457 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.719529 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.719514 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c3b48e53-1f3c-4143-8b10-766db3ea33fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.727675 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.727657 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg5rx\" (UniqueName: \"kubernetes.io/projected/c3b48e53-1f3c-4143-8b10-766db3ea33fc-kube-api-access-qg5rx\") pod \"opendatahub-operator-controller-manager-f8cc9fd78-wdfhw\" (UID: \"c3b48e53-1f3c-4143-8b10-766db3ea33fc\") " pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.781751 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.781731 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:17.911903 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.911872 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw"] Jun 02 02:52:17.915803 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:52:17.915779 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b48e53_1f3c_4143_8b10_766db3ea33fc.slice/crio-3bb0ff12134d2a6a112b8a7e2249bac75e7b5d1888ce39b595570368b3134476 WatchSource:0}: Error finding container 3bb0ff12134d2a6a112b8a7e2249bac75e7b5d1888ce39b595570368b3134476: Status 404 returned error can't find the container with id 3bb0ff12134d2a6a112b8a7e2249bac75e7b5d1888ce39b595570368b3134476 Jun 02 02:52:17.932364 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:17.932340 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" event={"ID":"c3b48e53-1f3c-4143-8b10-766db3ea33fc","Type":"ContainerStarted","Data":"3bb0ff12134d2a6a112b8a7e2249bac75e7b5d1888ce39b595570368b3134476"} Jun 02 02:52:20.945023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:20.944992 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" event={"ID":"c3b48e53-1f3c-4143-8b10-766db3ea33fc","Type":"ContainerStarted","Data":"b23b783963577161a618c18554ef767d82cab88355e5e0f0a370ca06bd9e853b"} Jun 02 02:52:21.949469 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:21.949440 2563 generic.go:358] "Generic (PLEG): container finished" podID="c3b48e53-1f3c-4143-8b10-766db3ea33fc" containerID="b23b783963577161a618c18554ef767d82cab88355e5e0f0a370ca06bd9e853b" exitCode=0 Jun 02 02:52:21.949849 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:21.949530 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" event={"ID":"c3b48e53-1f3c-4143-8b10-766db3ea33fc","Type":"ContainerDied","Data":"b23b783963577161a618c18554ef767d82cab88355e5e0f0a370ca06bd9e853b"} Jun 02 02:52:22.954764 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:22.954736 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" event={"ID":"c3b48e53-1f3c-4143-8b10-766db3ea33fc","Type":"ContainerStarted","Data":"81994d2c17ab7746d2756321e329f971ceea3a012a902275aecff09760e07aa0"} Jun 02 02:52:22.955179 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:22.954829 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:22.980272 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:22.980225 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" podStartSLOduration=1.5848270690000001 podStartE2EDuration="5.980210423s" podCreationTimestamp="2026-06-02 02:52:17 +0000 UTC" firstStartedPulling="2026-06-02 02:52:17.91739701 +0000 UTC m=+417.888997473" lastFinishedPulling="2026-06-02 02:52:22.312780361 +0000 UTC m=+422.284380827" observedRunningTime="2026-06-02 02:52:22.979262916 +0000 UTC m=+422.950863403" watchObservedRunningTime="2026-06-02 02:52:22.980210423 +0000 UTC m=+422.951810908" Jun 02 02:52:33.960771 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:33.960736 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f8cc9fd78-wdfhw" Jun 02 02:52:44.143927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.143853 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2"] Jun 02 02:52:44.147318 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.147301 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.150325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.150304 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 02:52:44.151383 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.151364 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bnsbg\"" Jun 02 02:52:44.154067 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.154052 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 02:52:44.154423 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.154407 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 02:52:44.168338 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.168316 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2"] Jun 02 02:52:44.225546 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.225523 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-manager-config\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.225659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.225553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.225659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.225590 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.225659 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.225635 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khv2v\" (UniqueName: \"kubernetes.io/projected/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-kube-api-access-khv2v\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.326910 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.326878 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.327024 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.326933 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-khv2v\" (UniqueName: \"kubernetes.io/projected/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-kube-api-access-khv2v\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.327087 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.327021 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-manager-config\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.327087 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.327052 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.327858 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.327832 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-manager-config\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.329530 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.329510 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-metrics-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.329614 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.329581 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-cert\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.335643 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.335619 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khv2v\" (UniqueName: \"kubernetes.io/projected/b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d-kube-api-access-khv2v\") pod \"lws-controller-manager-774f9bf4f6-kksl2\" (UID: \"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d\") " pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.456938 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.456867 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:44.588497 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:44.587240 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2"] Jun 02 02:52:44.591340 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:52:44.591312 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb881c2f6_e45d_4d9d_8a65_70cc6d1e1d4d.slice/crio-b3ddf0d71a43b2f327420927000fa69d8546ec2cc9116bc1f7286ce5a3688917 WatchSource:0}: Error finding container b3ddf0d71a43b2f327420927000fa69d8546ec2cc9116bc1f7286ce5a3688917: Status 404 returned error can't find the container with id b3ddf0d71a43b2f327420927000fa69d8546ec2cc9116bc1f7286ce5a3688917 Jun 02 02:52:45.037932 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:45.037904 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" event={"ID":"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d","Type":"ContainerStarted","Data":"b3ddf0d71a43b2f327420927000fa69d8546ec2cc9116bc1f7286ce5a3688917"} Jun 02 02:52:47.046999 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:47.046960 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" event={"ID":"b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d","Type":"ContainerStarted","Data":"ad622f7d218b262befdbda09ebb53710d498020034a724b3fd4f51f7f6246dd7"} Jun 02 02:52:47.047440 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:47.047092 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:52:47.114326 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:47.114281 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" podStartSLOduration=1.102674221 podStartE2EDuration="3.114267568s" podCreationTimestamp="2026-06-02 02:52:44 +0000 UTC" firstStartedPulling="2026-06-02 02:52:44.593172879 +0000 UTC m=+444.564773342" lastFinishedPulling="2026-06-02 02:52:46.604766214 +0000 UTC m=+446.576366689" observedRunningTime="2026-06-02 02:52:47.109983355 +0000 UTC m=+447.081583840" watchObservedRunningTime="2026-06-02 02:52:47.114267568 +0000 UTC m=+447.085868052" Jun 02 02:52:51.699332 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.699299 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9"] Jun 02 02:52:51.702961 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.702937 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.706513 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.706488 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 02:52:51.706638 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.706497 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bvb9s\"" Jun 02 02:52:51.716958 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.716933 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9"] Jun 02 02:52:51.798240 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.798195 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4qtd\" (UniqueName: \"kubernetes.io/projected/682ca309-f070-4a72-ba11-aa63638a6764-kube-api-access-z4qtd\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.798397 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.798305 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/682ca309-f070-4a72-ba11-aa63638a6764-redirect-config\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.899654 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.899614 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z4qtd\" (UniqueName: \"kubernetes.io/projected/682ca309-f070-4a72-ba11-aa63638a6764-kube-api-access-z4qtd\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.899817 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.899676 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/682ca309-f070-4a72-ba11-aa63638a6764-redirect-config\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.900446 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.900425 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/682ca309-f070-4a72-ba11-aa63638a6764-redirect-config\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:51.909974 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:51.909953 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4qtd\" (UniqueName: \"kubernetes.io/projected/682ca309-f070-4a72-ba11-aa63638a6764-kube-api-access-z4qtd\") pod \"dashboard-redirect-6d9fb5d8dc-zqvf9\" (UID: \"682ca309-f070-4a72-ba11-aa63638a6764\") " pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:52.013789 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:52.013725 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" Jun 02 02:52:52.145307 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:52.145276 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9"] Jun 02 02:52:52.147815 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:52:52.147790 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod682ca309_f070_4a72_ba11_aa63638a6764.slice/crio-17626e72088c85aa31b9ddc7eec64e3978378de3c02f0fe65765da8acb14c2f5 WatchSource:0}: Error finding container 17626e72088c85aa31b9ddc7eec64e3978378de3c02f0fe65765da8acb14c2f5: Status 404 returned error can't find the container with id 17626e72088c85aa31b9ddc7eec64e3978378de3c02f0fe65765da8acb14c2f5 Jun 02 02:52:53.067114 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:53.067072 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" event={"ID":"682ca309-f070-4a72-ba11-aa63638a6764","Type":"ContainerStarted","Data":"17626e72088c85aa31b9ddc7eec64e3978378de3c02f0fe65765da8acb14c2f5"} Jun 02 02:52:55.076257 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:55.076223 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" event={"ID":"682ca309-f070-4a72-ba11-aa63638a6764","Type":"ContainerStarted","Data":"c642ad1c5b3235e9616fcd00283bb1ebb11ca4727ff7d68384781b4dc8d72b87"} Jun 02 02:52:55.095272 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:55.095230 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6d9fb5d8dc-zqvf9" podStartSLOduration=2.022767821 podStartE2EDuration="4.095217638s" podCreationTimestamp="2026-06-02 02:52:51 +0000 UTC" firstStartedPulling="2026-06-02 02:52:52.149099922 +0000 UTC m=+452.120700386" lastFinishedPulling="2026-06-02 02:52:54.221549737 +0000 UTC m=+454.193150203" observedRunningTime="2026-06-02 02:52:55.094885263 +0000 UTC m=+455.066485746" watchObservedRunningTime="2026-06-02 02:52:55.095217638 +0000 UTC m=+455.066818124" Jun 02 02:52:58.052339 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:52:58.052307 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-774f9bf4f6-kksl2" Jun 02 02:53:01.475693 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.475662 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:01.480167 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.480133 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.482945 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.482905 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 02 02:53:01.483361 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.483343 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-79wlx\"" Jun 02 02:53:01.488988 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.483527 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 02:53:01.489201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.487697 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 02:53:01.491034 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.491013 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:01.576660 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576631 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576815 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576815 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576691 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576815 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs56\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576807 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576864 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.576927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.577023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576942 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.577023 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.576971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678214 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678173 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678366 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678231 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678366 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678278 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678366 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678319 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs56\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678366 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678363 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678565 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678410 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678565 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678454 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678565 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678481 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678565 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678507 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678753 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678709 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678809 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678760 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.678957 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678936 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.679029 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.678984 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.679029 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.679008 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.680631 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.680605 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.680723 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.680633 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.686332 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.686314 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs56\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.686479 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.686459 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token\") pod \"data-science-gateway-data-science-gateway-class-587dfcbc44bqncw\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.798220 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.798137 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:01.811586 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.811560 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6"] Jun 02 02:53:01.816663 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.816645 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.828480 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.828456 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6"] Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880084 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmp5\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-kube-api-access-9vmp5\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880139 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880267 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880324 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880420 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.880757 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.880428 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.928292 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.928262 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:01.931290 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:53:01.931265 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b3182f_5b9b_4345_bf2c_5816167ded33.slice/crio-6b6f48352f7003bb0831da4236668dd9360f923d24fef0323f5951d6110471a1 WatchSource:0}: Error finding container 6b6f48352f7003bb0831da4236668dd9360f923d24fef0323f5951d6110471a1: Status 404 returned error can't find the container with id 6b6f48352f7003bb0831da4236668dd9360f923d24fef0323f5951d6110471a1 Jun 02 02:53:01.980965 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.980942 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981065 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.980982 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981065 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981008 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981065 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981031 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981065 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981055 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981105 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981205 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981271 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981440 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981303 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmp5\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-kube-api-access-9vmp5\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981440 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981371 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981728 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981710 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981793 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981732 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.981875 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.981855 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.982231 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.982208 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.983359 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.983339 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.983589 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.983574 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.989553 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.989528 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:01.989774 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:01.989753 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmp5\" (UniqueName: \"kubernetes.io/projected/995f0e50-2d56-4e89-a047-36ddd51dd2b7-kube-api-access-9vmp5\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6\" (UID: \"995f0e50-2d56-4e89-a047-36ddd51dd2b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:02.099851 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:02.099778 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" event={"ID":"42b3182f-5b9b-4345-bf2c-5816167ded33","Type":"ContainerStarted","Data":"6b6f48352f7003bb0831da4236668dd9360f923d24fef0323f5951d6110471a1"} Jun 02 02:53:02.131061 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:02.131035 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:02.253726 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:02.253702 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6"] Jun 02 02:53:02.255612 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:53:02.255586 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995f0e50_2d56_4e89_a047_36ddd51dd2b7.slice/crio-e38e57ece8c8b2f2c7597c1d74c137a78446f77216f5c2872696adf00de0e55f WatchSource:0}: Error finding container e38e57ece8c8b2f2c7597c1d74c137a78446f77216f5c2872696adf00de0e55f: Status 404 returned error can't find the container with id e38e57ece8c8b2f2c7597c1d74c137a78446f77216f5c2872696adf00de0e55f Jun 02 02:53:03.106745 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:03.106704 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" event={"ID":"995f0e50-2d56-4e89-a047-36ddd51dd2b7","Type":"ContainerStarted","Data":"e38e57ece8c8b2f2c7597c1d74c137a78446f77216f5c2872696adf00de0e55f"} Jun 02 02:53:04.847548 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.847507 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:04.847927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.847576 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:04.847927 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.847603 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:04.853349 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.853324 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:04.853421 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.853377 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:04.853421 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:04.853406 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:53:05.116820 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.116722 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" event={"ID":"42b3182f-5b9b-4345-bf2c-5816167ded33","Type":"ContainerStarted","Data":"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c"} Jun 02 02:53:05.118265 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.118233 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" event={"ID":"995f0e50-2d56-4e89-a047-36ddd51dd2b7","Type":"ContainerStarted","Data":"0e4f4de94b25d6e07eb816abfc4a1e69d2823373e61d309a414a92735b2135bd"} Jun 02 02:53:05.131788 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.131762 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:05.132975 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.132954 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jun 02 02:53:05.133078 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.132995 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" podUID="995f0e50-2d56-4e89-a047-36ddd51dd2b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jun 02 02:53:05.137168 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.137101 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" podStartSLOduration=1.222929984 podStartE2EDuration="4.137084912s" podCreationTimestamp="2026-06-02 02:53:01 +0000 UTC" firstStartedPulling="2026-06-02 02:53:01.933120609 +0000 UTC m=+461.904721071" lastFinishedPulling="2026-06-02 02:53:04.847275532 +0000 UTC m=+464.818875999" observedRunningTime="2026-06-02 02:53:05.135426012 +0000 UTC m=+465.107026518" watchObservedRunningTime="2026-06-02 02:53:05.137084912 +0000 UTC m=+465.108685398" Jun 02 02:53:05.157105 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.157057 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" podStartSLOduration=1.561435312 podStartE2EDuration="4.157043608s" podCreationTimestamp="2026-06-02 02:53:01 +0000 UTC" firstStartedPulling="2026-06-02 02:53:02.257492229 +0000 UTC m=+462.229092691" lastFinishedPulling="2026-06-02 02:53:04.853100524 +0000 UTC m=+464.824700987" observedRunningTime="2026-06-02 02:53:05.154334568 +0000 UTC m=+465.125935054" watchObservedRunningTime="2026-06-02 02:53:05.157043608 +0000 UTC m=+465.128644096" Jun 02 02:53:05.798929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.798887 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:05.800293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.800268 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 02 02:53:05.800411 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:05.800315 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 02 02:53:06.135387 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:06.135359 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:06.799453 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:06.799422 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 02 02:53:06.799610 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:06.799474 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 02 02:53:07.132269 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:07.132245 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:07.133218 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:07.133198 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6" Jun 02 02:53:07.215561 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:07.215532 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:07.799405 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:07.799371 2563 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jun 02 02:53:07.799563 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:07.799432 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jun 02 02:53:08.135883 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:08.135851 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" containerID="cri-o://68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c" gracePeriod=30 Jun 02 02:53:13.416759 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.416737 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:13.588308 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588275 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588308 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588308 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588352 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588382 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588421 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588444 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588468 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588504 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcs56\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588810 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588549 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data\") pod \"42b3182f-5b9b-4345-bf2c-5816167ded33\" (UID: \"42b3182f-5b9b-4345-bf2c-5816167ded33\") " Jun 02 02:53:13.588810 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588578 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:53:13.588810 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588725 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:53:13.588810 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588796 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data" (OuterVolumeSpecName: "istio-data") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:53:13.589017 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588905 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:53:13.589017 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588951 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-certs\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.589017 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588971 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-credential-socket\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.589017 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.588988 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-data\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.589240 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.589165 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 02:53:13.590774 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.590748 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 02 02:53:13.590890 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.590790 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56" (OuterVolumeSpecName: "kube-api-access-tcs56") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "kube-api-access-tcs56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:53:13.590890 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.590830 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:53:13.590890 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.590866 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token" (OuterVolumeSpecName: "istio-token") pod "42b3182f-5b9b-4345-bf2c-5816167ded33" (UID: "42b3182f-5b9b-4345-bf2c-5816167ded33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690123 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-token\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690163 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-podinfo\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690173 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-istio-envoy\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690180 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b3182f-5b9b-4345-bf2c-5816167ded33-istiod-ca-cert\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690189 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b3182f-5b9b-4345-bf2c-5816167ded33-workload-socket\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:13.690198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:13.690197 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs56\" (UniqueName: \"kubernetes.io/projected/42b3182f-5b9b-4345-bf2c-5816167ded33-kube-api-access-tcs56\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:14.157221 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.157184 2563 generic.go:358] "Generic (PLEG): container finished" podID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerID="68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c" exitCode=0 Jun 02 02:53:14.157385 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.157253 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" Jun 02 02:53:14.157385 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.157278 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" event={"ID":"42b3182f-5b9b-4345-bf2c-5816167ded33","Type":"ContainerDied","Data":"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c"} Jun 02 02:53:14.157385 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.157326 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw" event={"ID":"42b3182f-5b9b-4345-bf2c-5816167ded33","Type":"ContainerDied","Data":"6b6f48352f7003bb0831da4236668dd9360f923d24fef0323f5951d6110471a1"} Jun 02 02:53:14.157385 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.157347 2563 scope.go:117] "RemoveContainer" containerID="68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c" Jun 02 02:53:14.168539 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.168522 2563 scope.go:117] "RemoveContainer" containerID="68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c" Jun 02 02:53:14.168791 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:53:14.168773 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c\": container with ID starting with 68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c not found: ID does not exist" containerID="68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c" Jun 02 02:53:14.168854 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.168805 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c"} err="failed to get container status \"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c\": rpc error: code = NotFound desc = could not find container \"68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c\": container with ID starting with 68a1f50df777732bb96f8f607a9c7a1d391aaebe9add4809cffbe19f9091307c not found: ID does not exist" Jun 02 02:53:14.183598 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.183578 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:14.189389 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.189364 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-587dfcbc44bqncw"] Jun 02 02:53:14.511219 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:14.511115 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" path="/var/lib/kubelet/pods/42b3182f-5b9b-4345-bf2c-5816167ded33/volumes" Jun 02 02:53:37.577561 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.577526 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:37.578013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.577881 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" Jun 02 02:53:37.578013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.577892 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" Jun 02 02:53:37.578013 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.577954 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="42b3182f-5b9b-4345-bf2c-5816167ded33" containerName="istio-proxy" Jun 02 02:53:37.594099 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.594076 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:37.600604 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.600516 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 02:53:37.600731 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.600664 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-f7v7x\"" Jun 02 02:53:37.601474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.601271 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 02:53:37.602672 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.602651 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:37.683710 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.683671 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm8xr\" (UniqueName: \"kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr\") pod \"kuadrant-operator-catalog-tlrpd\" (UID: \"8942dd0f-1457-456c-b1f0-fb9a2937067f\") " pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:37.784937 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.784898 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jm8xr\" (UniqueName: \"kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr\") pod \"kuadrant-operator-catalog-tlrpd\" (UID: \"8942dd0f-1457-456c-b1f0-fb9a2937067f\") " pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:37.795528 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.795500 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm8xr\" (UniqueName: \"kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr\") pod \"kuadrant-operator-catalog-tlrpd\" (UID: \"8942dd0f-1457-456c-b1f0-fb9a2937067f\") " pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:37.897879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.897846 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:37.898129 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:37.898108 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:38.021342 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.021319 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:38.023549 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:53:38.023518 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8942dd0f_1457_456c_b1f0_fb9a2937067f.slice/crio-c0454beccf3adaf7c8f8f375298de0cd242d55538255d95971fd1663387a1f5b WatchSource:0}: Error finding container c0454beccf3adaf7c8f8f375298de0cd242d55538255d95971fd1663387a1f5b: Status 404 returned error can't find the container with id c0454beccf3adaf7c8f8f375298de0cd242d55538255d95971fd1663387a1f5b Jun 02 02:53:38.106897 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.106864 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r72v9"] Jun 02 02:53:38.110630 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.110613 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:38.116892 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.116866 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r72v9"] Jun 02 02:53:38.189139 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.189061 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j564x\" (UniqueName: \"kubernetes.io/projected/36c949f6-71a2-41d5-ba05-53c2e2220b7a-kube-api-access-j564x\") pod \"kuadrant-operator-catalog-r72v9\" (UID: \"36c949f6-71a2-41d5-ba05-53c2e2220b7a\") " pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:38.239524 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.239494 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" event={"ID":"8942dd0f-1457-456c-b1f0-fb9a2937067f","Type":"ContainerStarted","Data":"c0454beccf3adaf7c8f8f375298de0cd242d55538255d95971fd1663387a1f5b"} Jun 02 02:53:38.289851 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.289825 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j564x\" (UniqueName: \"kubernetes.io/projected/36c949f6-71a2-41d5-ba05-53c2e2220b7a-kube-api-access-j564x\") pod \"kuadrant-operator-catalog-r72v9\" (UID: \"36c949f6-71a2-41d5-ba05-53c2e2220b7a\") " pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:38.298246 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.298215 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j564x\" (UniqueName: \"kubernetes.io/projected/36c949f6-71a2-41d5-ba05-53c2e2220b7a-kube-api-access-j564x\") pod \"kuadrant-operator-catalog-r72v9\" (UID: \"36c949f6-71a2-41d5-ba05-53c2e2220b7a\") " pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:38.421192 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.421159 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:38.554389 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:38.554366 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r72v9"] Jun 02 02:53:38.594364 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:53:38.594324 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c949f6_71a2_41d5_ba05_53c2e2220b7a.slice/crio-8dd49b6a18bd0ed984b5dcb9c112b0ab699a53474efedd275872e421e689a981 WatchSource:0}: Error finding container 8dd49b6a18bd0ed984b5dcb9c112b0ab699a53474efedd275872e421e689a981: Status 404 returned error can't find the container with id 8dd49b6a18bd0ed984b5dcb9c112b0ab699a53474efedd275872e421e689a981 Jun 02 02:53:39.246512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:39.246475 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" event={"ID":"36c949f6-71a2-41d5-ba05-53c2e2220b7a","Type":"ContainerStarted","Data":"8dd49b6a18bd0ed984b5dcb9c112b0ab699a53474efedd275872e421e689a981"} Jun 02 02:53:40.251808 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.251775 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" event={"ID":"8942dd0f-1457-456c-b1f0-fb9a2937067f","Type":"ContainerStarted","Data":"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131"} Jun 02 02:53:40.252263 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.251837 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" podUID="8942dd0f-1457-456c-b1f0-fb9a2937067f" containerName="registry-server" containerID="cri-o://8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131" gracePeriod=2 Jun 02 02:53:40.253233 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.253214 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" event={"ID":"36c949f6-71a2-41d5-ba05-53c2e2220b7a","Type":"ContainerStarted","Data":"3e67dd67ace1659e0b4dcea327114299cfd48176cb3225504b79f65a092a40c3"} Jun 02 02:53:40.282879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.282828 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" podStartSLOduration=0.753105458 podStartE2EDuration="2.282809642s" podCreationTimestamp="2026-06-02 02:53:38 +0000 UTC" firstStartedPulling="2026-06-02 02:53:38.595642731 +0000 UTC m=+498.567243194" lastFinishedPulling="2026-06-02 02:53:40.125346911 +0000 UTC m=+500.096947378" observedRunningTime="2026-06-02 02:53:40.281579929 +0000 UTC m=+500.253180426" watchObservedRunningTime="2026-06-02 02:53:40.282809642 +0000 UTC m=+500.254410130" Jun 02 02:53:40.283354 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.283321 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" podStartSLOduration=1.185747384 podStartE2EDuration="3.283310809s" podCreationTimestamp="2026-06-02 02:53:37 +0000 UTC" firstStartedPulling="2026-06-02 02:53:38.024960068 +0000 UTC m=+497.996560545" lastFinishedPulling="2026-06-02 02:53:40.122523497 +0000 UTC m=+500.094123970" observedRunningTime="2026-06-02 02:53:40.267251067 +0000 UTC m=+500.238851554" watchObservedRunningTime="2026-06-02 02:53:40.283310809 +0000 UTC m=+500.254911293" Jun 02 02:53:40.498321 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.498299 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tlrpd_8942dd0f-1457-456c-b1f0-fb9a2937067f/registry-server/0.log" Jun 02 02:53:40.498417 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.498355 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:40.509573 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.509552 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm8xr\" (UniqueName: \"kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr\") pod \"8942dd0f-1457-456c-b1f0-fb9a2937067f\" (UID: \"8942dd0f-1457-456c-b1f0-fb9a2937067f\") " Jun 02 02:53:40.511997 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.511971 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr" (OuterVolumeSpecName: "kube-api-access-jm8xr") pod "8942dd0f-1457-456c-b1f0-fb9a2937067f" (UID: "8942dd0f-1457-456c-b1f0-fb9a2937067f"). InnerVolumeSpecName "kube-api-access-jm8xr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:53:40.610644 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:40.610615 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm8xr\" (UniqueName: \"kubernetes.io/projected/8942dd0f-1457-456c-b1f0-fb9a2937067f-kube-api-access-jm8xr\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:53:41.257876 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.257848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-tlrpd_8942dd0f-1457-456c-b1f0-fb9a2937067f/registry-server/0.log" Jun 02 02:53:41.258346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.257891 2563 generic.go:358] "Generic (PLEG): container finished" podID="8942dd0f-1457-456c-b1f0-fb9a2937067f" containerID="8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131" exitCode=1 Jun 02 02:53:41.258346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.257951 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" Jun 02 02:53:41.258346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.257968 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" event={"ID":"8942dd0f-1457-456c-b1f0-fb9a2937067f","Type":"ContainerDied","Data":"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131"} Jun 02 02:53:41.258346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.258001 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tlrpd" event={"ID":"8942dd0f-1457-456c-b1f0-fb9a2937067f","Type":"ContainerDied","Data":"c0454beccf3adaf7c8f8f375298de0cd242d55538255d95971fd1663387a1f5b"} Jun 02 02:53:41.258346 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.258015 2563 scope.go:117] "RemoveContainer" containerID="8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131" Jun 02 02:53:41.266950 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.266933 2563 scope.go:117] "RemoveContainer" containerID="8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131" Jun 02 02:53:41.267206 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:53:41.267187 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131\": container with ID starting with 8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131 not found: ID does not exist" containerID="8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131" Jun 02 02:53:41.267271 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.267218 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131"} err="failed to get container status \"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131\": rpc error: code = NotFound desc = could not find container \"8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131\": container with ID starting with 8a0da68116bc2f4149416bef81e37678f99fe8908646b610b0e30e278a828131 not found: ID does not exist" Jun 02 02:53:41.278273 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.278251 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:41.281728 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:41.281709 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tlrpd"] Jun 02 02:53:42.511180 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:42.511122 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8942dd0f-1457-456c-b1f0-fb9a2937067f" path="/var/lib/kubelet/pods/8942dd0f-1457-456c-b1f0-fb9a2937067f/volumes" Jun 02 02:53:44.774652 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.774616 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-75ccdbdd7b-6jhjn"] Jun 02 02:53:44.775052 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.775035 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8942dd0f-1457-456c-b1f0-fb9a2937067f" containerName="registry-server" Jun 02 02:53:44.775119 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.775054 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8942dd0f-1457-456c-b1f0-fb9a2937067f" containerName="registry-server" Jun 02 02:53:44.775199 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.775159 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="8942dd0f-1457-456c-b1f0-fb9a2937067f" containerName="registry-server" Jun 02 02:53:44.779362 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.779344 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.781895 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.781871 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 02:53:44.781895 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.781884 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 02:53:44.783047 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.783026 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 02:53:44.783047 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.783051 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 02:53:44.783264 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.783100 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rcdq4\"" Jun 02 02:53:44.783264 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.783049 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 02:53:44.788196 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.788177 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 02:53:44.789046 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.789025 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ccdbdd7b-6jhjn"] Jun 02 02:53:44.847812 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.847929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcvnq\" (UniqueName: \"kubernetes.io/projected/3b92604a-e661-4ba7-a5aa-28e7079c5be2-kube-api-access-gcvnq\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.847929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.847929 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-oauth-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.848042 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847959 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-service-ca\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.848042 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.847982 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-oauth-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.848042 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.848017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-trusted-ca-bundle\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949198 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949169 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949205 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gcvnq\" (UniqueName: \"kubernetes.io/projected/3b92604a-e661-4ba7-a5aa-28e7079c5be2-kube-api-access-gcvnq\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949225 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949260 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-oauth-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949290 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-service-ca\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949312 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-oauth-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.949581 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.949332 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-trusted-ca-bundle\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.950031 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.950007 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.950031 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.950025 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-service-ca\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.950240 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.950031 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-oauth-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.950302 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.950240 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b92604a-e661-4ba7-a5aa-28e7079c5be2-trusted-ca-bundle\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.951698 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.951677 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-oauth-config\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.951798 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.951782 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b92604a-e661-4ba7-a5aa-28e7079c5be2-console-serving-cert\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:44.956574 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:44.956553 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcvnq\" (UniqueName: \"kubernetes.io/projected/3b92604a-e661-4ba7-a5aa-28e7079c5be2-kube-api-access-gcvnq\") pod \"console-75ccdbdd7b-6jhjn\" (UID: \"3b92604a-e661-4ba7-a5aa-28e7079c5be2\") " pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:45.091283 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:45.091201 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:45.227620 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:45.227591 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75ccdbdd7b-6jhjn"] Jun 02 02:53:45.229088 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:53:45.229062 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b92604a_e661_4ba7_a5aa_28e7079c5be2.slice/crio-cdf83a3efa9d6e4ad560c61f0cab4e2efacb72ac14cc80a63aa58e6c196faee5 WatchSource:0}: Error finding container cdf83a3efa9d6e4ad560c61f0cab4e2efacb72ac14cc80a63aa58e6c196faee5: Status 404 returned error can't find the container with id cdf83a3efa9d6e4ad560c61f0cab4e2efacb72ac14cc80a63aa58e6c196faee5 Jun 02 02:53:45.274903 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:45.274875 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccdbdd7b-6jhjn" event={"ID":"3b92604a-e661-4ba7-a5aa-28e7079c5be2","Type":"ContainerStarted","Data":"cdf83a3efa9d6e4ad560c61f0cab4e2efacb72ac14cc80a63aa58e6c196faee5"} Jun 02 02:53:46.279683 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:46.279640 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75ccdbdd7b-6jhjn" event={"ID":"3b92604a-e661-4ba7-a5aa-28e7079c5be2","Type":"ContainerStarted","Data":"fc968aa636d9b29f3f9371290654af8f083a33628d1b3fa4e9b9f67310eb1b9f"} Jun 02 02:53:46.299098 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:46.299043 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-75ccdbdd7b-6jhjn" podStartSLOduration=2.299029525 podStartE2EDuration="2.299029525s" podCreationTimestamp="2026-06-02 02:53:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:53:46.296750113 +0000 UTC m=+506.268350601" watchObservedRunningTime="2026-06-02 02:53:46.299029525 +0000 UTC m=+506.270630009" Jun 02 02:53:48.421503 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:48.421461 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:48.421503 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:48.421502 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:48.443242 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:48.443215 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:49.313976 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:49.313948 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r72v9" Jun 02 02:53:55.092083 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:55.092052 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:55.092487 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:55.092093 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:55.096806 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:55.096785 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:53:55.317591 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:53:55.317564 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75ccdbdd7b-6jhjn" Jun 02 02:54:10.955778 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:10.955698 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-gkmj9"] Jun 02 02:54:10.959136 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:10.959118 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:10.963267 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:10.963248 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7mplx\"" Jun 02 02:54:10.970632 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:10.970606 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-gkmj9"] Jun 02 02:54:11.075735 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:11.075706 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjm8r\" (UniqueName: \"kubernetes.io/projected/759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf-kube-api-access-gjm8r\") pod \"authorino-operator-657f44b778-gkmj9\" (UID: \"759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf\") " pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:11.176120 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:11.176088 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjm8r\" (UniqueName: \"kubernetes.io/projected/759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf-kube-api-access-gjm8r\") pod \"authorino-operator-657f44b778-gkmj9\" (UID: \"759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf\") " pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:11.184983 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:11.184953 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjm8r\" (UniqueName: \"kubernetes.io/projected/759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf-kube-api-access-gjm8r\") pod \"authorino-operator-657f44b778-gkmj9\" (UID: \"759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf\") " pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:11.270371 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:11.270313 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:11.415494 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:11.415474 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-gkmj9"] Jun 02 02:54:11.417967 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:11.417945 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod759e4a6b_e6ff_4ae7_b089_e5f7c711c2bf.slice/crio-9404b6bbf97b0fb953a7952a2c16a5483dae7eab5fff507c4f0b3acf26a7c663 WatchSource:0}: Error finding container 9404b6bbf97b0fb953a7952a2c16a5483dae7eab5fff507c4f0b3acf26a7c663: Status 404 returned error can't find the container with id 9404b6bbf97b0fb953a7952a2c16a5483dae7eab5fff507c4f0b3acf26a7c663 Jun 02 02:54:12.374957 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:12.374922 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" event={"ID":"759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf","Type":"ContainerStarted","Data":"9404b6bbf97b0fb953a7952a2c16a5483dae7eab5fff507c4f0b3acf26a7c663"} Jun 02 02:54:14.384564 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:14.384527 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" event={"ID":"759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf","Type":"ContainerStarted","Data":"a876a5ac2264e23ee8c3a6f5eec7afb5bbf1bf7a53608da7de1bdce32e071dd0"} Jun 02 02:54:14.384935 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:14.384662 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:14.403049 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:14.403006 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" podStartSLOduration=2.145533781 podStartE2EDuration="4.402991604s" podCreationTimestamp="2026-06-02 02:54:10 +0000 UTC" firstStartedPulling="2026-06-02 02:54:11.422726447 +0000 UTC m=+531.394326910" lastFinishedPulling="2026-06-02 02:54:13.680184261 +0000 UTC m=+533.651784733" observedRunningTime="2026-06-02 02:54:14.400677276 +0000 UTC m=+534.372277761" watchObservedRunningTime="2026-06-02 02:54:14.402991604 +0000 UTC m=+534.374592088" Jun 02 02:54:16.049733 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.049694 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk"] Jun 02 02:54:16.053482 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.053459 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.056007 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.055978 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b9plv\"" Jun 02 02:54:16.064706 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.064681 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk"] Jun 02 02:54:16.226962 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.226923 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.227175 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.226985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbt2q\" (UniqueName: \"kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.327486 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.327422 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.327486 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.327466 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hbt2q\" (UniqueName: \"kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.327824 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.327804 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.336643 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.336621 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbt2q\" (UniqueName: \"kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.364755 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.364728 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:16.495226 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:16.495203 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk"] Jun 02 02:54:16.497515 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:16.497489 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b01080_d7bd_4008_8525_c8231732efcc.slice/crio-4ea876e411c7e98717b50a96761d989f72feb37448c69b32145f6e47769e544f WatchSource:0}: Error finding container 4ea876e411c7e98717b50a96761d989f72feb37448c69b32145f6e47769e544f: Status 404 returned error can't find the container with id 4ea876e411c7e98717b50a96761d989f72feb37448c69b32145f6e47769e544f Jun 02 02:54:17.398684 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:17.398646 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" event={"ID":"95b01080-d7bd-4008-8525-c8231732efcc","Type":"ContainerStarted","Data":"4ea876e411c7e98717b50a96761d989f72feb37448c69b32145f6e47769e544f"} Jun 02 02:54:21.417509 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.417474 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" event={"ID":"95b01080-d7bd-4008-8525-c8231732efcc","Type":"ContainerStarted","Data":"74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd"} Jun 02 02:54:21.417962 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.417627 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:21.438535 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.438455 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" podStartSLOduration=0.984340828 podStartE2EDuration="5.438440238s" podCreationTimestamp="2026-06-02 02:54:16 +0000 UTC" firstStartedPulling="2026-06-02 02:54:16.499873585 +0000 UTC m=+536.471474048" lastFinishedPulling="2026-06-02 02:54:20.953972983 +0000 UTC m=+540.925573458" observedRunningTime="2026-06-02 02:54:21.435637664 +0000 UTC m=+541.407238149" watchObservedRunningTime="2026-06-02 02:54:21.438440238 +0000 UTC m=+541.410040723" Jun 02 02:54:21.491571 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.491542 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc"] Jun 02 02:54:21.495108 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.495092 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:21.498693 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.498674 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c969j\"" Jun 02 02:54:21.498932 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.498909 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 02:54:21.504740 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.504719 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc"] Jun 02 02:54:21.577507 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.577474 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k492t\" (UniqueName: \"kubernetes.io/projected/8c1e0625-15c7-47da-bdeb-959bfcca1db0-kube-api-access-k492t\") pod \"dns-operator-controller-manager-648d5c98bc-bwhtc\" (UID: \"8c1e0625-15c7-47da-bdeb-959bfcca1db0\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:21.678789 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.678702 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k492t\" (UniqueName: \"kubernetes.io/projected/8c1e0625-15c7-47da-bdeb-959bfcca1db0-kube-api-access-k492t\") pod \"dns-operator-controller-manager-648d5c98bc-bwhtc\" (UID: \"8c1e0625-15c7-47da-bdeb-959bfcca1db0\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:21.693590 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.693556 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k492t\" (UniqueName: \"kubernetes.io/projected/8c1e0625-15c7-47da-bdeb-959bfcca1db0-kube-api-access-k492t\") pod \"dns-operator-controller-manager-648d5c98bc-bwhtc\" (UID: \"8c1e0625-15c7-47da-bdeb-959bfcca1db0\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:21.806390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.806357 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:21.939838 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:21.939814 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc"] Jun 02 02:54:21.941888 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:21.941864 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c1e0625_15c7_47da_bdeb_959bfcca1db0.slice/crio-ab7b7fec670a0dc2834d97f83c3bf6b750d4ae9e9a5b2f687f762c4586e09bd5 WatchSource:0}: Error finding container ab7b7fec670a0dc2834d97f83c3bf6b750d4ae9e9a5b2f687f762c4586e09bd5: Status 404 returned error can't find the container with id ab7b7fec670a0dc2834d97f83c3bf6b750d4ae9e9a5b2f687f762c4586e09bd5 Jun 02 02:54:22.422400 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:22.422365 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" event={"ID":"8c1e0625-15c7-47da-bdeb-959bfcca1db0","Type":"ContainerStarted","Data":"ab7b7fec670a0dc2834d97f83c3bf6b750d4ae9e9a5b2f687f762c4586e09bd5"} Jun 02 02:54:24.430411 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:24.430371 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" event={"ID":"8c1e0625-15c7-47da-bdeb-959bfcca1db0","Type":"ContainerStarted","Data":"b963134ded6454f41c9937c1916f808acae96a33cbfc58f5b420a29c02600e16"} Jun 02 02:54:24.430866 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:24.430486 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:24.447166 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:24.447096 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" podStartSLOduration=1.4080161869999999 podStartE2EDuration="3.447076552s" podCreationTimestamp="2026-06-02 02:54:21 +0000 UTC" firstStartedPulling="2026-06-02 02:54:21.944250037 +0000 UTC m=+541.915850500" lastFinishedPulling="2026-06-02 02:54:23.983310399 +0000 UTC m=+543.954910865" observedRunningTime="2026-06-02 02:54:24.445743734 +0000 UTC m=+544.417344221" watchObservedRunningTime="2026-06-02 02:54:24.447076552 +0000 UTC m=+544.418677099" Jun 02 02:54:25.390003 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:25.389975 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-gkmj9" Jun 02 02:54:32.425185 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:32.425129 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:33.803767 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.803736 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw"] Jun 02 02:54:33.807590 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.807570 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:33.820268 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.820246 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw"] Jun 02 02:54:33.892725 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.892700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bl7\" (UniqueName: \"kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:33.892869 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.892746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:33.994041 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.994013 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bl7\" (UniqueName: \"kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:33.994194 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.994059 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:33.994427 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:33.994409 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:34.012531 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.012505 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bl7\" (UniqueName: \"kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7\") pod \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:34.117315 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.117235 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:34.237451 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.237419 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk"] Jun 02 02:54:34.237647 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.237625 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" podUID="95b01080-d7bd-4008-8525-c8231732efcc" containerName="manager" containerID="cri-o://74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd" gracePeriod=2 Jun 02 02:54:34.247050 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.246434 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw"] Jun 02 02:54:34.249040 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.249012 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk"] Jun 02 02:54:34.250747 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:34.250706 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb7ebcc0_6766_4f1d_a20a_a363b0472dd9.slice/crio-4f314872b13f3b4199c2970d89c071c375ed51557b5b2df2bafa6fe26bc93712 WatchSource:0}: Error finding container 4f314872b13f3b4199c2970d89c071c375ed51557b5b2df2bafa6fe26bc93712: Status 404 returned error can't find the container with id 4f314872b13f3b4199c2970d89c071c375ed51557b5b2df2bafa6fe26bc93712 Jun 02 02:54:34.257751 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.257700 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw"] Jun 02 02:54:34.259828 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.259802 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:34.260344 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.260295 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b01080-d7bd-4008-8525-c8231732efcc" containerName="manager" Jun 02 02:54:34.263171 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.263132 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b01080-d7bd-4008-8525-c8231732efcc" containerName="manager" Jun 02 02:54:34.263462 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.263447 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="95b01080-d7bd-4008-8525-c8231732efcc" containerName="manager" Jun 02 02:54:34.266914 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.266892 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw"] Jun 02 02:54:34.267069 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.267020 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.269985 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.269946 2563 status_manager.go:895] "Failed to get status for pod" podUID="95b01080-d7bd-4008-8525-c8231732efcc" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.273947 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.273927 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:34.282986 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.282884 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:54:34.287202 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.287186 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.301769 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.301745 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:54:34.328758 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.328728 2563 status_manager.go:895] "Failed to get status for pod" podUID="95b01080-d7bd-4008-8525-c8231732efcc" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.396725 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.396702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r56r\" (UniqueName: \"kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.396848 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.396736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.396848 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.396761 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcrnl\" (UniqueName: \"kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.396848 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.396793 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.469319 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.469294 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:34.472474 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.472450 2563 status_manager.go:895] "Failed to get status for pod" podUID="95b01080-d7bd-4008-8525-c8231732efcc" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.481237 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.481194 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" containerName="manager" containerID="cri-o://5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8" gracePeriod=2 Jun 02 02:54:34.482500 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.482476 2563 generic.go:358] "Generic (PLEG): container finished" podID="95b01080-d7bd-4008-8525-c8231732efcc" containerID="74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd" exitCode=0 Jun 02 02:54:34.482598 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.482519 2563 scope.go:117] "RemoveContainer" containerID="74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd" Jun 02 02:54:34.482598 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.482534 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" Jun 02 02:54:34.485726 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.485701 2563 status_manager.go:895] "Failed to get status for pod" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" err="pods \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.487976 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.487951 2563 status_manager.go:895] "Failed to get status for pod" podUID="95b01080-d7bd-4008-8525-c8231732efcc" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.490525 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.490505 2563 status_manager.go:895] "Failed to get status for pod" podUID="95b01080-d7bd-4008-8525-c8231732efcc" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-tqqgk" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-tqqgk\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.490687 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.490668 2563 scope.go:117] "RemoveContainer" containerID="74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd" Jun 02 02:54:34.490926 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:54:34.490911 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd\": container with ID starting with 74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd not found: ID does not exist" containerID="74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd" Jun 02 02:54:34.490963 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.490934 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd"} err="failed to get container status \"74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd\": rpc error: code = NotFound desc = could not find container \"74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd\": container with ID starting with 74d8ad7f50edd4f6b1d40420fcc2ff154765cdcd094f4086535fb9399c4874cd not found: ID does not exist" Jun 02 02:54:34.492676 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.492656 2563 status_manager.go:895] "Failed to get status for pod" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" err="pods \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:34.498102 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498081 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zcrnl\" (UniqueName: \"kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.498191 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498130 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.498234 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498205 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8r56r\" (UniqueName: \"kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.498234 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498228 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.498521 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498504 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.498562 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.498519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.523623 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.523602 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r56r\" (UniqueName: \"kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-cpr8l\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.523623 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.523618 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcrnl\" (UniqueName: \"kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl\") pod \"kuadrant-operator-controller-manager-84b657d985-mtk5d\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.599028 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.599004 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume\") pod \"95b01080-d7bd-4008-8525-c8231732efcc\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " Jun 02 02:54:34.599136 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.599081 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbt2q\" (UniqueName: \"kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q\") pod \"95b01080-d7bd-4008-8525-c8231732efcc\" (UID: \"95b01080-d7bd-4008-8525-c8231732efcc\") " Jun 02 02:54:34.599512 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.599483 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "95b01080-d7bd-4008-8525-c8231732efcc" (UID: "95b01080-d7bd-4008-8525-c8231732efcc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:54:34.600886 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.600865 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q" (OuterVolumeSpecName: "kube-api-access-hbt2q") pod "95b01080-d7bd-4008-8525-c8231732efcc" (UID: "95b01080-d7bd-4008-8525-c8231732efcc"). InnerVolumeSpecName "kube-api-access-hbt2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:54:34.643717 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.643651 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:34.666173 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.666132 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:34.700048 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.699980 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/95b01080-d7bd-4008-8525-c8231732efcc-extensions-socket-volume\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:34.700048 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.700018 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbt2q\" (UniqueName: \"kubernetes.io/projected/95b01080-d7bd-4008-8525-c8231732efcc-kube-api-access-hbt2q\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:34.712767 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.712680 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:34.794767 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.794746 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:34.797480 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:34.797454 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d3076a_f120_47d2_85f2_7e4fd43ba5fa.slice/crio-562ee36689979ef74a268c0991a95cbb7e81aefe8c2c4f2a2ca0fe5a59d46c6d WatchSource:0}: Error finding container 562ee36689979ef74a268c0991a95cbb7e81aefe8c2c4f2a2ca0fe5a59d46c6d: Status 404 returned error can't find the container with id 562ee36689979ef74a268c0991a95cbb7e81aefe8c2c4f2a2ca0fe5a59d46c6d Jun 02 02:54:34.800285 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.800259 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume\") pod \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " Jun 02 02:54:34.800434 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.800382 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5bl7\" (UniqueName: \"kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7\") pod \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\" (UID: \"fb7ebcc0-6766-4f1d-a20a-a363b0472dd9\") " Jun 02 02:54:34.800497 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.800478 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" (UID: "fb7ebcc0-6766-4f1d-a20a-a363b0472dd9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:54:34.800652 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.800635 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-extensions-socket-volume\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:34.802272 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.802248 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7" (OuterVolumeSpecName: "kube-api-access-c5bl7") pod "fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" (UID: "fb7ebcc0-6766-4f1d-a20a-a363b0472dd9"). InnerVolumeSpecName "kube-api-access-c5bl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:54:34.811596 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.811578 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:54:34.813552 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:34.813531 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d601f4c_b5c8_45a2_b839_afaa0f027006.slice/crio-7d8b493367d8b033ec8fa89a7fd2331c342f49f15737404ac92945033ecc3f36 WatchSource:0}: Error finding container 7d8b493367d8b033ec8fa89a7fd2331c342f49f15737404ac92945033ecc3f36: Status 404 returned error can't find the container with id 7d8b493367d8b033ec8fa89a7fd2331c342f49f15737404ac92945033ecc3f36 Jun 02 02:54:34.902195 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:34.902113 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5bl7\" (UniqueName: \"kubernetes.io/projected/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9-kube-api-access-c5bl7\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:35.443426 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.443398 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bwhtc" Jun 02 02:54:35.488994 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.488956 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" event={"ID":"3d601f4c-b5c8-45a2-b839-afaa0f027006","Type":"ContainerStarted","Data":"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648"} Jun 02 02:54:35.488994 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.488994 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" event={"ID":"3d601f4c-b5c8-45a2-b839-afaa0f027006","Type":"ContainerStarted","Data":"7d8b493367d8b033ec8fa89a7fd2331c342f49f15737404ac92945033ecc3f36"} Jun 02 02:54:35.489210 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.489089 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:35.490185 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.490135 2563 generic.go:358] "Generic (PLEG): container finished" podID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" containerID="5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8" exitCode=2 Jun 02 02:54:35.490292 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.490191 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" Jun 02 02:54:35.490351 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.490190 2563 scope.go:117] "RemoveContainer" containerID="5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8" Jun 02 02:54:35.491790 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.491763 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" event={"ID":"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa","Type":"ContainerStarted","Data":"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd"} Jun 02 02:54:35.491872 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.491793 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" event={"ID":"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa","Type":"ContainerStarted","Data":"562ee36689979ef74a268c0991a95cbb7e81aefe8c2c4f2a2ca0fe5a59d46c6d"} Jun 02 02:54:35.491936 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.491877 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:35.499295 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.499217 2563 scope.go:117] "RemoveContainer" containerID="5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8" Jun 02 02:54:35.499492 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:54:35.499467 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8\": container with ID starting with 5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8 not found: ID does not exist" containerID="5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8" Jun 02 02:54:35.499543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.499492 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8"} err="failed to get container status \"5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8\": rpc error: code = NotFound desc = could not find container \"5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8\": container with ID starting with 5b497d818802fec321af738ab1ed713dde545b1eae6d00908e91f009552788c8 not found: ID does not exist" Jun 02 02:54:35.509581 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.509535 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" podStartSLOduration=1.509520492 podStartE2EDuration="1.509520492s" podCreationTimestamp="2026-06-02 02:54:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:54:35.507963825 +0000 UTC m=+555.479564321" watchObservedRunningTime="2026-06-02 02:54:35.509520492 +0000 UTC m=+555.481120978" Jun 02 02:54:35.509824 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.509796 2563 status_manager.go:895] "Failed to get status for pod" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-fkqhw" err="pods \"kuadrant-operator-controller-manager-84b657d985-fkqhw\" is forbidden: User \"system:node:ip-10-0-138-19.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-19.ec2.internal' and this object" Jun 02 02:54:35.527118 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:35.527062 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" podStartSLOduration=1.527052122 podStartE2EDuration="1.527052122s" podCreationTimestamp="2026-06-02 02:54:34 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:54:35.526359225 +0000 UTC m=+555.497959712" watchObservedRunningTime="2026-06-02 02:54:35.527052122 +0000 UTC m=+555.498652608" Jun 02 02:54:36.510280 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:36.510246 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95b01080-d7bd-4008-8525-c8231732efcc" path="/var/lib/kubelet/pods/95b01080-d7bd-4008-8525-c8231732efcc/volumes" Jun 02 02:54:36.510640 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:36.510574 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" path="/var/lib/kubelet/pods/fb7ebcc0-6766-4f1d-a20a-a363b0472dd9/volumes" Jun 02 02:54:46.499329 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.499299 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:46.499833 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.499351 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:46.564293 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.564256 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:46.564482 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.564458 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" podUID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" containerName="manager" containerID="cri-o://767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd" gracePeriod=10 Jun 02 02:54:46.812606 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.812581 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:46.843585 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.843559 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9"] Jun 02 02:54:46.844127 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844097 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" containerName="manager" Jun 02 02:54:46.844127 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844117 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" containerName="manager" Jun 02 02:54:46.844286 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844172 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" containerName="manager" Jun 02 02:54:46.844286 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844180 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" containerName="manager" Jun 02 02:54:46.844286 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844269 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="fb7ebcc0-6766-4f1d-a20a-a363b0472dd9" containerName="manager" Jun 02 02:54:46.844286 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.844282 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" containerName="manager" Jun 02 02:54:46.848390 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.848370 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:46.854245 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.854224 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9"] Jun 02 02:54:46.904211 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.904185 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r56r\" (UniqueName: \"kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r\") pod \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " Jun 02 02:54:46.904337 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.904323 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume\") pod \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\" (UID: \"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa\") " Jun 02 02:54:46.904700 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.904681 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" (UID: "c8d3076a-f120-47d2-85f2-7e4fd43ba5fa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:54:46.905984 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:46.905961 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r" (OuterVolumeSpecName: "kube-api-access-8r56r") pod "c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" (UID: "c8d3076a-f120-47d2-85f2-7e4fd43ba5fa"). InnerVolumeSpecName "kube-api-access-8r56r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:54:47.005076 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.005042 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1de53d41-da72-47e1-acc4-3203e5bb0968-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.005233 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.005138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d62vr\" (UniqueName: \"kubernetes.io/projected/1de53d41-da72-47e1-acc4-3203e5bb0968-kube-api-access-d62vr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.005288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.005232 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-extensions-socket-volume\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:47.005288 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.005244 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r56r\" (UniqueName: \"kubernetes.io/projected/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa-kube-api-access-8r56r\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:47.105944 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.105856 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d62vr\" (UniqueName: \"kubernetes.io/projected/1de53d41-da72-47e1-acc4-3203e5bb0968-kube-api-access-d62vr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.105944 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.105938 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1de53d41-da72-47e1-acc4-3203e5bb0968-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.106361 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.106342 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1de53d41-da72-47e1-acc4-3203e5bb0968-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.117061 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.117035 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d62vr\" (UniqueName: \"kubernetes.io/projected/1de53d41-da72-47e1-acc4-3203e5bb0968-kube-api-access-d62vr\") pod \"kuadrant-operator-controller-manager-55c7f4c975-d96k9\" (UID: \"1de53d41-da72-47e1-acc4-3203e5bb0968\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.160435 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.160414 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.313912 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.313888 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9"] Jun 02 02:54:47.316314 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:54:47.316282 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1de53d41_da72_47e1_acc4_3203e5bb0968.slice/crio-2f1e3f0be517cd1fe3920c6ac42273246da75898792ab0e760125690f4550532 WatchSource:0}: Error finding container 2f1e3f0be517cd1fe3920c6ac42273246da75898792ab0e760125690f4550532: Status 404 returned error can't find the container with id 2f1e3f0be517cd1fe3920c6ac42273246da75898792ab0e760125690f4550532 Jun 02 02:54:47.538691 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.538660 2563 generic.go:358] "Generic (PLEG): container finished" podID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" containerID="767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd" exitCode=0 Jun 02 02:54:47.539176 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.538718 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" Jun 02 02:54:47.539176 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.538749 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" event={"ID":"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa","Type":"ContainerDied","Data":"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd"} Jun 02 02:54:47.539176 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.538789 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l" event={"ID":"c8d3076a-f120-47d2-85f2-7e4fd43ba5fa","Type":"ContainerDied","Data":"562ee36689979ef74a268c0991a95cbb7e81aefe8c2c4f2a2ca0fe5a59d46c6d"} Jun 02 02:54:47.539176 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.538808 2563 scope.go:117] "RemoveContainer" containerID="767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd" Jun 02 02:54:47.540490 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.540460 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" event={"ID":"1de53d41-da72-47e1-acc4-3203e5bb0968","Type":"ContainerStarted","Data":"b1be26b0cb5b87a46d42a22d89ab63f95cdde3eaebd880f4e25380336450b539"} Jun 02 02:54:47.540490 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.540486 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" event={"ID":"1de53d41-da72-47e1-acc4-3203e5bb0968","Type":"ContainerStarted","Data":"2f1e3f0be517cd1fe3920c6ac42273246da75898792ab0e760125690f4550532"} Jun 02 02:54:47.540624 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.540573 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:47.548278 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.548259 2563 scope.go:117] "RemoveContainer" containerID="767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd" Jun 02 02:54:47.548516 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:54:47.548497 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd\": container with ID starting with 767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd not found: ID does not exist" containerID="767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd" Jun 02 02:54:47.548574 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.548527 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd"} err="failed to get container status \"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd\": rpc error: code = NotFound desc = could not find container \"767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd\": container with ID starting with 767b0f2ad8927e3ca0b5287d1dba31c87743052a518a7419220c37d9b3cc6bfd not found: ID does not exist" Jun 02 02:54:47.570209 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.570100 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" podStartSLOduration=1.5700850979999998 podStartE2EDuration="1.570085098s" podCreationTimestamp="2026-06-02 02:54:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:54:47.568759707 +0000 UTC m=+567.540360191" watchObservedRunningTime="2026-06-02 02:54:47.570085098 +0000 UTC m=+567.541685584" Jun 02 02:54:47.585434 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.585411 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:47.591223 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:47.591203 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-cpr8l"] Jun 02 02:54:48.512244 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:48.512214 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8d3076a-f120-47d2-85f2-7e4fd43ba5fa" path="/var/lib/kubelet/pods/c8d3076a-f120-47d2-85f2-7e4fd43ba5fa/volumes" Jun 02 02:54:58.546892 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:58.546858 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-d96k9" Jun 02 02:54:58.599259 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:58.599225 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:54:58.599493 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:58.599455 2563 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" podUID="3d601f4c-b5c8-45a2-b839-afaa0f027006" containerName="manager" containerID="cri-o://cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648" gracePeriod=10 Jun 02 02:54:58.838049 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:58.838027 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:59.008256 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.008220 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcrnl\" (UniqueName: \"kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl\") pod \"3d601f4c-b5c8-45a2-b839-afaa0f027006\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " Jun 02 02:54:59.008449 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.008265 2563 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume\") pod \"3d601f4c-b5c8-45a2-b839-afaa0f027006\" (UID: \"3d601f4c-b5c8-45a2-b839-afaa0f027006\") " Jun 02 02:54:59.008651 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.008620 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3d601f4c-b5c8-45a2-b839-afaa0f027006" (UID: "3d601f4c-b5c8-45a2-b839-afaa0f027006"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 02:54:59.010310 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.010286 2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl" (OuterVolumeSpecName: "kube-api-access-zcrnl") pod "3d601f4c-b5c8-45a2-b839-afaa0f027006" (UID: "3d601f4c-b5c8-45a2-b839-afaa0f027006"). InnerVolumeSpecName "kube-api-access-zcrnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 02:54:59.109883 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.109814 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcrnl\" (UniqueName: \"kubernetes.io/projected/3d601f4c-b5c8-45a2-b839-afaa0f027006-kube-api-access-zcrnl\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:59.109883 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.109839 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d601f4c-b5c8-45a2-b839-afaa0f027006-extensions-socket-volume\") on node \"ip-10-0-138-19.ec2.internal\" DevicePath \"\"" Jun 02 02:54:59.588678 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.588648 2563 generic.go:358] "Generic (PLEG): container finished" podID="3d601f4c-b5c8-45a2-b839-afaa0f027006" containerID="cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648" exitCode=0 Jun 02 02:54:59.589058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.588706 2563 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" Jun 02 02:54:59.589058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.588734 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" event={"ID":"3d601f4c-b5c8-45a2-b839-afaa0f027006","Type":"ContainerDied","Data":"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648"} Jun 02 02:54:59.589058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.588773 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d" event={"ID":"3d601f4c-b5c8-45a2-b839-afaa0f027006","Type":"ContainerDied","Data":"7d8b493367d8b033ec8fa89a7fd2331c342f49f15737404ac92945033ecc3f36"} Jun 02 02:54:59.589058 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.588793 2563 scope.go:117] "RemoveContainer" containerID="cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648" Jun 02 02:54:59.598137 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.598126 2563 scope.go:117] "RemoveContainer" containerID="cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648" Jun 02 02:54:59.598407 ip-10-0-138-19 kubenswrapper[2563]: E0602 02:54:59.598390 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648\": container with ID starting with cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648 not found: ID does not exist" containerID="cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648" Jun 02 02:54:59.598446 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.598415 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648"} err="failed to get container status \"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648\": rpc error: code = NotFound desc = could not find container \"cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648\": container with ID starting with cb1b1729f1e1c5059485a948aac9b8dc4b9e3e3e5e9059b22284124101ffd648 not found: ID does not exist" Jun 02 02:54:59.621218 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.621195 2563 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:54:59.624329 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:54:59.624309 2563 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-mtk5d"] Jun 02 02:55:00.510883 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:00.510855 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d601f4c-b5c8-45a2-b839-afaa0f027006" path="/var/lib/kubelet/pods/3d601f4c-b5c8-45a2-b839-afaa0f027006/volumes" Jun 02 02:55:02.885199 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.885170 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595"] Jun 02 02:55:02.885609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.885595 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d601f4c-b5c8-45a2-b839-afaa0f027006" containerName="manager" Jun 02 02:55:02.885656 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.885610 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d601f4c-b5c8-45a2-b839-afaa0f027006" containerName="manager" Jun 02 02:55:02.885691 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.885683 2563 memory_manager.go:356] "RemoveStaleState removing state" podUID="3d601f4c-b5c8-45a2-b839-afaa0f027006" containerName="manager" Jun 02 02:55:02.890987 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.890962 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.893881 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.893863 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kzgzm\"" Jun 02 02:55:02.903334 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.903310 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595"] Jun 02 02:55:02.938879 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.938857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.938978 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.938893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.938978 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.938918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dvfp\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-kube-api-access-8dvfp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939054 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.938981 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939054 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.939039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939126 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.939064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939126 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.939086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939126 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.939113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:02.939268 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:02.939191 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040072 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040048 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040082 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040106 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040131 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040194 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040265 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040301 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040336 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8dvfp\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-kube-api-access-8dvfp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040394 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040362 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040585 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040519 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040693 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040672 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040796 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040774 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.040859 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.040806 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.041066 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.041044 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.042428 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.042408 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.042755 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.042740 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.050438 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.050415 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.050971 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.050951 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dvfp\" (UniqueName: \"kubernetes.io/projected/4d3ee16b-779a-4fac-83df-9ed85c9763fb-kube-api-access-8dvfp\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-69595\" (UID: \"4d3ee16b-779a-4fac-83df-9ed85c9763fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.204552 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.204488 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:03.339300 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.339273 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595"] Jun 02 02:55:03.340601 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:55:03.340572 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d3ee16b_779a_4fac_83df_9ed85c9763fb.slice/crio-c1401c5ec7d6ade4d9841604ae2d8c72a9bb4a97ab864c4d1664652ca9babdf5 WatchSource:0}: Error finding container c1401c5ec7d6ade4d9841604ae2d8c72a9bb4a97ab864c4d1664652ca9babdf5: Status 404 returned error can't find the container with id c1401c5ec7d6ade4d9841604ae2d8c72a9bb4a97ab864c4d1664652ca9babdf5 Jun 02 02:55:03.342626 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.342591 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:55:03.342719 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.342650 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:55:03.342719 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.342681 2563 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 02 02:55:03.607694 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.607618 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" event={"ID":"4d3ee16b-779a-4fac-83df-9ed85c9763fb","Type":"ContainerStarted","Data":"ff98b143823e20add2d805d6b1cfc062c693895b1b29ed317ba6d817024f80dd"} Jun 02 02:55:03.607694 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.607657 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" event={"ID":"4d3ee16b-779a-4fac-83df-9ed85c9763fb","Type":"ContainerStarted","Data":"c1401c5ec7d6ade4d9841604ae2d8c72a9bb4a97ab864c4d1664652ca9babdf5"} Jun 02 02:55:03.627312 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:03.627262 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" podStartSLOduration=1.6272494929999999 podStartE2EDuration="1.627249493s" podCreationTimestamp="2026-06-02 02:55:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 02:55:03.625451616 +0000 UTC m=+583.597052101" watchObservedRunningTime="2026-06-02 02:55:03.627249493 +0000 UTC m=+583.598849977" Jun 02 02:55:04.204864 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:04.204833 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:04.209543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:04.209521 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:04.611543 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:04.611469 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:04.612528 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:04.612504 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-69595" Jun 02 02:55:16.879609 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.879576 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:55:16.883152 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.883127 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:16.886008 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.885983 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mcdtr\"" Jun 02 02:55:16.886169 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.886052 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 02:55:16.892959 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.892934 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:55:16.916410 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.916389 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:55:16.967552 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.967524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4d2x\" (UniqueName: \"kubernetes.io/projected/848afa29-1876-487d-b71e-7f7c3020a65d-kube-api-access-l4d2x\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:16.967654 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:16.967625 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/848afa29-1876-487d-b71e-7f7c3020a65d-config-file\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.068294 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.068267 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/848afa29-1876-487d-b71e-7f7c3020a65d-config-file\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.068403 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.068319 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l4d2x\" (UniqueName: \"kubernetes.io/projected/848afa29-1876-487d-b71e-7f7c3020a65d-kube-api-access-l4d2x\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.068852 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.068833 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/848afa29-1876-487d-b71e-7f7c3020a65d-config-file\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.076642 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.076621 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4d2x\" (UniqueName: \"kubernetes.io/projected/848afa29-1876-487d-b71e-7f7c3020a65d-kube-api-access-l4d2x\") pod \"limitador-limitador-78c99df468-s4d2q\" (UID: \"848afa29-1876-487d-b71e-7f7c3020a65d\") " pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.194698 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.194628 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:17.327047 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.327021 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:55:17.329373 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:55:17.329345 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848afa29_1876_487d_b71e_7f7c3020a65d.slice/crio-f077b02efcf272b31200520a5cafdbdb2edebb0f49a9df3786cde7fe44a5a316 WatchSource:0}: Error finding container f077b02efcf272b31200520a5cafdbdb2edebb0f49a9df3786cde7fe44a5a316: Status 404 returned error can't find the container with id f077b02efcf272b31200520a5cafdbdb2edebb0f49a9df3786cde7fe44a5a316 Jun 02 02:55:17.660472 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:17.660434 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" event={"ID":"848afa29-1876-487d-b71e-7f7c3020a65d","Type":"ContainerStarted","Data":"f077b02efcf272b31200520a5cafdbdb2edebb0f49a9df3786cde7fe44a5a316"} Jun 02 02:55:20.428276 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.428251 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:55:20.428708 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.428483 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 02:55:20.430712 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.430692 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:55:20.430837 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.430789 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 02:55:20.673226 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.673185 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" event={"ID":"848afa29-1876-487d-b71e-7f7c3020a65d","Type":"ContainerStarted","Data":"f9bb5a838676127455b5ef8d0eba8d6903ebf957269b71761f9a0ce88ae05af2"} Jun 02 02:55:20.673381 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.673275 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:55:20.689925 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.689844 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" podStartSLOduration=2.30655054 podStartE2EDuration="4.689832173s" podCreationTimestamp="2026-06-02 02:55:16 +0000 UTC" firstStartedPulling="2026-06-02 02:55:17.331067228 +0000 UTC m=+597.302667692" lastFinishedPulling="2026-06-02 02:55:19.714348859 +0000 UTC m=+599.685949325" observedRunningTime="2026-06-02 02:55:20.688410425 +0000 UTC m=+600.660010911" watchObservedRunningTime="2026-06-02 02:55:20.689832173 +0000 UTC m=+600.661432657" Jun 02 02:55:20.902120 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:20.902093 2563 scope.go:117] "RemoveContainer" containerID="7e4e57b975101ab2214a0b30d0a585f26ed38a6669b29579fa61c00f049ec3f5" Jun 02 02:55:31.678667 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:55:31.678635 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-s4d2q" Jun 02 02:57:17.807185 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.807099 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-9gzvs"] Jun 02 02:57:17.810877 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.810860 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:17.813407 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.813387 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 02 02:57:17.813407 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.813395 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wv2kk\"" Jun 02 02:57:17.823973 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.823950 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-9gzvs"] Jun 02 02:57:17.894819 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.894797 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:57:17.904478 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:17.904456 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5flmx\" (UniqueName: \"kubernetes.io/projected/0a219430-e140-48e4-9e71-d06c92f6f277-kube-api-access-5flmx\") pod \"payload-processing-58b6c8fdc7-9gzvs\" (UID: \"0a219430-e140-48e4-9e71-d06c92f6f277\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:18.006015 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:18.005971 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5flmx\" (UniqueName: \"kubernetes.io/projected/0a219430-e140-48e4-9e71-d06c92f6f277-kube-api-access-5flmx\") pod \"payload-processing-58b6c8fdc7-9gzvs\" (UID: \"0a219430-e140-48e4-9e71-d06c92f6f277\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:18.014354 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:18.014325 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5flmx\" (UniqueName: \"kubernetes.io/projected/0a219430-e140-48e4-9e71-d06c92f6f277-kube-api-access-5flmx\") pod \"payload-processing-58b6c8fdc7-9gzvs\" (UID: \"0a219430-e140-48e4-9e71-d06c92f6f277\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:18.121048 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:18.120973 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:18.252489 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:18.252443 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-9gzvs"] Jun 02 02:57:18.254560 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:57:18.254530 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a219430_e140_48e4_9e71_d06c92f6f277.slice/crio-4b7d377f963073fc3de3e7904dd828ac5fe770f8d77cfb482585641edd9f46d8 WatchSource:0}: Error finding container 4b7d377f963073fc3de3e7904dd828ac5fe770f8d77cfb482585641edd9f46d8: Status 404 returned error can't find the container with id 4b7d377f963073fc3de3e7904dd828ac5fe770f8d77cfb482585641edd9f46d8 Jun 02 02:57:18.256403 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:18.256386 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 02:57:19.100303 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:19.100264 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" event={"ID":"0a219430-e140-48e4-9e71-d06c92f6f277","Type":"ContainerStarted","Data":"4b7d377f963073fc3de3e7904dd828ac5fe770f8d77cfb482585641edd9f46d8"} Jun 02 02:57:20.939309 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:20.939280 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 02 02:57:21.109429 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:21.109400 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" event={"ID":"0a219430-e140-48e4-9e71-d06c92f6f277","Type":"ContainerStarted","Data":"c1397883cc1009f8de496090cf77c6e3bc0af143c3f0eb11bd00462ba396a9fd"} Jun 02 02:57:21.109574 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:21.109482 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:57:21.125819 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:21.125778 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" podStartSLOduration=1.44560364 podStartE2EDuration="4.125765166s" podCreationTimestamp="2026-06-02 02:57:17 +0000 UTC" firstStartedPulling="2026-06-02 02:57:18.256546363 +0000 UTC m=+718.228146826" lastFinishedPulling="2026-06-02 02:57:20.936707879 +0000 UTC m=+720.908308352" observedRunningTime="2026-06-02 02:57:21.123422769 +0000 UTC m=+721.095023248" watchObservedRunningTime="2026-06-02 02:57:21.125765166 +0000 UTC m=+721.097365650" Jun 02 02:57:32.120690 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:57:32.120658 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-9gzvs" Jun 02 02:58:14.157643 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.157614 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58"] Jun 02 02:58:14.161403 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.161387 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.164088 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.164066 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 02:58:14.165242 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.165223 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 02:58:14.165331 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.165222 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-84zz9\"" Jun 02 02:58:14.165331 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.165224 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 02 02:58:14.170624 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.170602 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58"] Jun 02 02:58:14.314274 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314244 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.314431 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.314431 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314307 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df8c6dd3-1918-4408-aa13-38760002ca4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.314431 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.314535 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.314535 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.314479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cp96\" (UniqueName: \"kubernetes.io/projected/df8c6dd3-1918-4408-aa13-38760002ca4b-kube-api-access-6cp96\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.415955 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.415870 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.415955 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.415910 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.415955 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.415930 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df8c6dd3-1918-4408-aa13-38760002ca4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416235 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.415971 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416235 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.416003 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416235 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.416032 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6cp96\" (UniqueName: \"kubernetes.io/projected/df8c6dd3-1918-4408-aa13-38760002ca4b-kube-api-access-6cp96\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416397 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.416358 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416456 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.416431 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.416550 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.416527 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.418334 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.418308 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df8c6dd3-1918-4408-aa13-38760002ca4b-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.419042 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.418540 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df8c6dd3-1918-4408-aa13-38760002ca4b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.423426 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.423406 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cp96\" (UniqueName: \"kubernetes.io/projected/df8c6dd3-1918-4408-aa13-38760002ca4b-kube-api-access-6cp96\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58\" (UID: \"df8c6dd3-1918-4408-aa13-38760002ca4b\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.472322 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.472297 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:14.620716 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.620688 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:58:14.624200 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:14.624180 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58"] Jun 02 02:58:14.626615 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:58:14.626590 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8c6dd3_1918_4408_aa13_38760002ca4b.slice/crio-bc24121bc001d3a24f706eb2d7eb749e155d0808eb7e8c2d19a1f0b12296c5fb WatchSource:0}: Error finding container bc24121bc001d3a24f706eb2d7eb749e155d0808eb7e8c2d19a1f0b12296c5fb: Status 404 returned error can't find the container with id bc24121bc001d3a24f706eb2d7eb749e155d0808eb7e8c2d19a1f0b12296c5fb Jun 02 02:58:15.314917 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:15.314875 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" event={"ID":"df8c6dd3-1918-4408-aa13-38760002ca4b","Type":"ContainerStarted","Data":"bc24121bc001d3a24f706eb2d7eb749e155d0808eb7e8c2d19a1f0b12296c5fb"} Jun 02 02:58:18.331367 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:18.331329 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" event={"ID":"df8c6dd3-1918-4408-aa13-38760002ca4b","Type":"ContainerStarted","Data":"0791dcc6789d97b57c35be9468b65bfd47962f5cdcde7e7917c8eee628fcce21"} Jun 02 02:58:20.339939 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:20.339905 2563 generic.go:358] "Generic (PLEG): container finished" podID="df8c6dd3-1918-4408-aa13-38760002ca4b" containerID="0791dcc6789d97b57c35be9468b65bfd47962f5cdcde7e7917c8eee628fcce21" exitCode=0 Jun 02 02:58:20.340369 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:20.339975 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" event={"ID":"df8c6dd3-1918-4408-aa13-38760002ca4b","Type":"ContainerDied","Data":"0791dcc6789d97b57c35be9468b65bfd47962f5cdcde7e7917c8eee628fcce21"} Jun 02 02:58:22.349501 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:22.349466 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" event={"ID":"df8c6dd3-1918-4408-aa13-38760002ca4b","Type":"ContainerStarted","Data":"9235c3b4020dadd31b45a6343d0b726a363c3fc0c288f5b3bb6ea554e4ae749d"} Jun 02 02:58:22.368105 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:22.368053 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" podStartSLOduration=1.633869669 podStartE2EDuration="8.368038878s" podCreationTimestamp="2026-06-02 02:58:14 +0000 UTC" firstStartedPulling="2026-06-02 02:58:14.628559596 +0000 UTC m=+774.600160059" lastFinishedPulling="2026-06-02 02:58:21.362728793 +0000 UTC m=+781.334329268" observedRunningTime="2026-06-02 02:58:22.36716505 +0000 UTC m=+782.338765534" watchObservedRunningTime="2026-06-02 02:58:22.368038878 +0000 UTC m=+782.339639363" Jun 02 02:58:24.473429 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:24.473395 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:24.473896 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:24.473446 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:24.485914 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:24.485890 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:24.683209 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:24.683173 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:58:25.371506 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:25.371473 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58" Jun 02 02:58:44.545799 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.545708 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn"] Jun 02 02:58:44.549526 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.549504 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.552461 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.552441 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 02:58:44.564287 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.564263 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn"] Jun 02 02:58:44.696201 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696167 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.696343 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696203 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzg5r\" (UniqueName: \"kubernetes.io/projected/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kube-api-access-vzg5r\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.696343 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696230 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.696343 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696291 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.696343 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.696540 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.696379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ac7f4b-abf7-4aeb-97c2-a323995e399e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798161 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798040 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798351 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798223 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vzg5r\" (UniqueName: \"kubernetes.io/projected/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kube-api-access-vzg5r\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798351 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798284 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798461 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798362 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798461 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798434 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798563 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798513 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ac7f4b-abf7-4aeb-97c2-a323995e399e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.798987 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.798963 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.799137 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.799042 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.799485 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.799463 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.801501 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.801474 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81ac7f4b-abf7-4aeb-97c2-a323995e399e-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.806361 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.806340 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81ac7f4b-abf7-4aeb-97c2-a323995e399e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.807325 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.807306 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzg5r\" (UniqueName: \"kubernetes.io/projected/81ac7f4b-abf7-4aeb-97c2-a323995e399e-kube-api-access-vzg5r\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn\" (UID: \"81ac7f4b-abf7-4aeb-97c2-a323995e399e\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.863922 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.863895 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:44.993647 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:44.993619 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn"] Jun 02 02:58:44.995301 ip-10-0-138-19 kubenswrapper[2563]: W0602 02:58:44.995275 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ac7f4b_abf7_4aeb_97c2_a323995e399e.slice/crio-9ebee75014ce5e72fa3a10893ae811ddd744b904ec22c7f7f168436caa09c23c WatchSource:0}: Error finding container 9ebee75014ce5e72fa3a10893ae811ddd744b904ec22c7f7f168436caa09c23c: Status 404 returned error can't find the container with id 9ebee75014ce5e72fa3a10893ae811ddd744b904ec22c7f7f168436caa09c23c Jun 02 02:58:45.435508 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:45.435473 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" event={"ID":"81ac7f4b-abf7-4aeb-97c2-a323995e399e","Type":"ContainerStarted","Data":"abbb850bb297122e55e57a34debf95bbd731d5f723b152972138321a0f42162b"} Jun 02 02:58:45.435508 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:45.435514 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" event={"ID":"81ac7f4b-abf7-4aeb-97c2-a323995e399e","Type":"ContainerStarted","Data":"9ebee75014ce5e72fa3a10893ae811ddd744b904ec22c7f7f168436caa09c23c"} Jun 02 02:58:45.878064 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:45.878038 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:58:46.441562 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:46.441525 2563 generic.go:358] "Generic (PLEG): container finished" podID="81ac7f4b-abf7-4aeb-97c2-a323995e399e" containerID="abbb850bb297122e55e57a34debf95bbd731d5f723b152972138321a0f42162b" exitCode=0 Jun 02 02:58:46.441697 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:46.441616 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" event={"ID":"81ac7f4b-abf7-4aeb-97c2-a323995e399e","Type":"ContainerDied","Data":"abbb850bb297122e55e57a34debf95bbd731d5f723b152972138321a0f42162b"} Jun 02 02:58:47.447464 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:47.447423 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" event={"ID":"81ac7f4b-abf7-4aeb-97c2-a323995e399e","Type":"ContainerStarted","Data":"7882a05742460d4ca31dbe0462e792fb384966ec47696a59091886859b613daa"} Jun 02 02:58:47.466386 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:47.466292 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" podStartSLOduration=3.280755765 podStartE2EDuration="3.466271349s" podCreationTimestamp="2026-06-02 02:58:44 +0000 UTC" firstStartedPulling="2026-06-02 02:58:46.442334703 +0000 UTC m=+806.413935180" lastFinishedPulling="2026-06-02 02:58:46.627850295 +0000 UTC m=+806.599450764" observedRunningTime="2026-06-02 02:58:47.464481674 +0000 UTC m=+807.436082174" watchObservedRunningTime="2026-06-02 02:58:47.466271349 +0000 UTC m=+807.437871837" Jun 02 02:58:54.279378 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:54.279343 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:58:54.864323 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:54.864289 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:54.864486 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:54.864439 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:54.877174 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:54.877124 2563 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:58:55.489280 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:58:55.489253 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn" Jun 02 02:59:09.283062 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:59:09.283029 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 02:59:17.114298 ip-10-0-138-19 kubenswrapper[2563]: I0602 02:59:17.114267 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:00:20.457643 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:20.457573 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 03:00:20.458735 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:20.458713 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 03:00:20.463766 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:20.463748 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 03:00:20.464580 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:20.464561 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 03:00:45.595060 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:45.595022 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:00:52.089412 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:00:52.089374 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:00.189603 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:00.189564 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:07.191931 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:07.191900 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:16.389876 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:16.389844 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:25.586895 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:25.586863 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:34.681972 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:34.681938 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:01:46.486990 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:01:46.486900 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:02:57.283838 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:02:57.283802 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:03:12.687905 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:03:12.687823 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:03:51.382051 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:03:51.382016 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:04:08.183794 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:08.183754 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:04:22.885830 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:22.885791 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:04:39.200998 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:39.200913 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-s4d2q"] Jun 02 03:04:56.636620 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:56.636579 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6d9fb5d8dc-zqvf9_682ca309-f070-4a72-ba11-aa63638a6764/nginx/0.log" Jun 02 03:04:57.863398 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:57.863369 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-wdfhw_c3b48e53-1f3c-4143-8b10-766db3ea33fc/copy-manifests/0.log" Jun 02 03:04:57.875598 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:57.875560 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-wdfhw_c3b48e53-1f3c-4143-8b10-766db3ea33fc/manager/0.log" Jun 02 03:04:59.428793 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:59.428764 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-gkmj9_759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf/manager/0.log" Jun 02 03:04:59.560977 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:59.560953 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bwhtc_8c1e0625-15c7-47da-bdeb-959bfcca1db0/manager/0.log" Jun 02 03:04:59.794363 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:59.794293 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r72v9_36c949f6-71a2-41d5-ba05-53c2e2220b7a/registry-server/0.log" Jun 02 03:04:59.923172 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:04:59.923129 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-d96k9_1de53d41-da72-47e1-acc4-3203e5bb0968/manager/0.log" Jun 02 03:05:00.037402 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:00.037375 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-s4d2q_848afa29-1876-487d-b71e-7f7c3020a65d/limitador/0.log" Jun 02 03:05:00.513882 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:00.513836 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6_995f0e50-2d56-4e89-a047-36ddd51dd2b7/istio-proxy/0.log" Jun 02 03:05:00.985857 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:00.985825 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-69595_4d3ee16b-779a-4fac-83df-9ed85c9763fb/istio-proxy/0.log" Jun 02 03:05:01.113263 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:01.113232 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-9gzvs_0a219430-e140-48e4-9e71-d06c92f6f277/payload-processing/0.log" Jun 02 03:05:01.616009 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:01.615976 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58_df8c6dd3-1918-4408-aa13-38760002ca4b/main/0.log" Jun 02 03:05:01.622890 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:01.622871 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-9pm58_df8c6dd3-1918-4408-aa13-38760002ca4b/storage-initializer/0.log" Jun 02 03:05:01.864717 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:01.864691 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn_81ac7f4b-abf7-4aeb-97c2-a323995e399e/storage-initializer/0.log" Jun 02 03:05:01.873633 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:01.873615 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-vk2gn_81ac7f4b-abf7-4aeb-97c2-a323995e399e/main/0.log" Jun 02 03:05:06.358282 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.358246 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sfk5k/must-gather-lxcbs"] Jun 02 03:05:06.362090 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.362074 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.364851 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.364826 2563 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sfk5k\"/\"default-dockercfg-8xdm6\"" Jun 02 03:05:06.365109 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.365094 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sfk5k\"/\"openshift-service-ca.crt\"" Jun 02 03:05:06.365768 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.365747 2563 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sfk5k\"/\"kube-root-ca.crt\"" Jun 02 03:05:06.400187 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.400164 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/must-gather-lxcbs"] Jun 02 03:05:06.435508 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.435479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wprw6\" (UniqueName: \"kubernetes.io/projected/b4b4ff4d-690d-44be-b14e-eae93c224e49-kube-api-access-wprw6\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.435627 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.435519 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b4b4ff4d-690d-44be-b14e-eae93c224e49-must-gather-output\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.536303 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.536269 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wprw6\" (UniqueName: \"kubernetes.io/projected/b4b4ff4d-690d-44be-b14e-eae93c224e49-kube-api-access-wprw6\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.536471 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.536309 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b4b4ff4d-690d-44be-b14e-eae93c224e49-must-gather-output\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.536593 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.536576 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b4b4ff4d-690d-44be-b14e-eae93c224e49-must-gather-output\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.557871 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.557846 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wprw6\" (UniqueName: \"kubernetes.io/projected/b4b4ff4d-690d-44be-b14e-eae93c224e49-kube-api-access-wprw6\") pod \"must-gather-lxcbs\" (UID: \"b4b4ff4d-690d-44be-b14e-eae93c224e49\") " pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.671413 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.671383 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" Jun 02 03:05:06.815195 ip-10-0-138-19 kubenswrapper[2563]: W0602 03:05:06.815166 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4b4ff4d_690d_44be_b14e_eae93c224e49.slice/crio-f68b99d535b346587f9d47741396ac6b7b7b363b0e1903755506a2b512e92f93 WatchSource:0}: Error finding container f68b99d535b346587f9d47741396ac6b7b7b363b0e1903755506a2b512e92f93: Status 404 returned error can't find the container with id f68b99d535b346587f9d47741396ac6b7b7b363b0e1903755506a2b512e92f93 Jun 02 03:05:06.817341 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.817323 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 03:05:06.819424 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.819402 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/must-gather-lxcbs"] Jun 02 03:05:06.856863 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:06.856835 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" event={"ID":"b4b4ff4d-690d-44be-b14e-eae93c224e49","Type":"ContainerStarted","Data":"f68b99d535b346587f9d47741396ac6b7b7b363b0e1903755506a2b512e92f93"} Jun 02 03:05:08.867944 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:08.867028 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" event={"ID":"b4b4ff4d-690d-44be-b14e-eae93c224e49","Type":"ContainerStarted","Data":"2c8404d35b6d30ac4365a1a67d5f94e5b08f18fcaf1fa66f44b6733cb275e245"} Jun 02 03:05:08.867944 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:08.867078 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" event={"ID":"b4b4ff4d-690d-44be-b14e-eae93c224e49","Type":"ContainerStarted","Data":"a61668760be666c71a8b90357792ac407db838137354cbc2ba1915c3942f1b4e"} Jun 02 03:05:08.888978 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:08.888930 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-sfk5k/must-gather-lxcbs" podStartSLOduration=1.932612486 podStartE2EDuration="2.888916556s" podCreationTimestamp="2026-06-02 03:05:06 +0000 UTC" firstStartedPulling="2026-06-02 03:05:06.817476674 +0000 UTC m=+1186.789077138" lastFinishedPulling="2026-06-02 03:05:07.773780743 +0000 UTC m=+1187.745381208" observedRunningTime="2026-06-02 03:05:08.887304373 +0000 UTC m=+1188.858904871" watchObservedRunningTime="2026-06-02 03:05:08.888916556 +0000 UTC m=+1188.860517041" Jun 02 03:05:09.786966 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:09.786936 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qrl6f_d8a3dee4-d352-4d4a-a453-62cea3c0ce58/global-pull-secret-syncer/0.log" Jun 02 03:05:09.914877 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:09.914845 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2zfx6_f16ec42c-bc37-4478-8319-ea326c7f5ed6/konnectivity-agent/0.log" Jun 02 03:05:10.057372 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:10.057274 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-19.ec2.internal_9f223c2f1cad1315f5d568b3ba3c4623/haproxy/0.log" Jun 02 03:05:13.916333 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:13.916294 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-gkmj9_759e4a6b-e6ff-4ae7-b089-e5f7c711c2bf/manager/0.log" Jun 02 03:05:13.972424 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:13.972396 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bwhtc_8c1e0625-15c7-47da-bdeb-959bfcca1db0/manager/0.log" Jun 02 03:05:14.054091 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:14.054062 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r72v9_36c949f6-71a2-41d5-ba05-53c2e2220b7a/registry-server/0.log" Jun 02 03:05:14.133248 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:14.133210 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-d96k9_1de53d41-da72-47e1-acc4-3203e5bb0968/manager/0.log" Jun 02 03:05:14.169042 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:14.168970 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-s4d2q_848afa29-1876-487d-b71e-7f7c3020a65d/limitador/0.log" Jun 02 03:05:15.728284 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:15.728208 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/alertmanager/0.log" Jun 02 03:05:15.797700 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:15.797669 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/config-reloader/0.log" Jun 02 03:05:15.862363 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:15.862332 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/kube-rbac-proxy-web/0.log" Jun 02 03:05:15.901992 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:15.901962 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/kube-rbac-proxy/0.log" Jun 02 03:05:15.953873 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:15.953799 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/kube-rbac-proxy-metric/0.log" Jun 02 03:05:16.012928 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.012837 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/prom-label-proxy/0.log" Jun 02 03:05:16.082667 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.082635 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d05b5cb2-31ca-4802-8bd2-ad63ba7c352b/init-config-reloader/0.log" Jun 02 03:05:16.185339 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.185312 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-5hsh8_d9b95afc-6853-4376-970b-d6412e5a8421/kube-state-metrics/0.log" Jun 02 03:05:16.224845 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.224810 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-5hsh8_d9b95afc-6853-4376-970b-d6412e5a8421/kube-rbac-proxy-main/0.log" Jun 02 03:05:16.274716 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.274631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-8469c486bb-5hsh8_d9b95afc-6853-4376-970b-d6412e5a8421/kube-rbac-proxy-self/0.log" Jun 02 03:05:16.379662 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.379630 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-npcbk_b260d2de-4531-4fda-a124-cb4ea40525e1/monitoring-plugin/0.log" Jun 02 03:05:16.425872 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.425846 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jdlx_52c03fa6-a0b6-40b7-8fc3-552ae559fbb7/node-exporter/0.log" Jun 02 03:05:16.460982 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.460956 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jdlx_52c03fa6-a0b6-40b7-8fc3-552ae559fbb7/kube-rbac-proxy/0.log" Jun 02 03:05:16.506231 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.506207 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6jdlx_52c03fa6-a0b6-40b7-8fc3-552ae559fbb7/init-textfile/0.log" Jun 02 03:05:16.807045 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.807010 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-czxf8_a53b89d2-403b-4b13-8c31-48c75074776a/kube-rbac-proxy-main/0.log" Jun 02 03:05:16.845991 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.845960 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-czxf8_a53b89d2-403b-4b13-8c31-48c75074776a/kube-rbac-proxy-self/0.log" Jun 02 03:05:16.888675 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.888644 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-85658796db-czxf8_a53b89d2-403b-4b13-8c31-48c75074776a/openshift-state-metrics/0.log" Jun 02 03:05:16.945741 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.945718 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/prometheus/0.log" Jun 02 03:05:16.986630 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:16.986558 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/config-reloader/0.log" Jun 02 03:05:17.029668 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.029641 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/thanos-sidecar/0.log" Jun 02 03:05:17.071907 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.071880 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/kube-rbac-proxy-web/0.log" Jun 02 03:05:17.139792 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.139764 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/kube-rbac-proxy/0.log" Jun 02 03:05:17.186306 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.186276 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/kube-rbac-proxy-thanos/0.log" Jun 02 03:05:17.232905 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.232862 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef09f87c-2e5e-4d78-9af3-d67df134c489/init-config-reloader/0.log" Jun 02 03:05:17.343085 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.343002 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-6brbc_a3043791-f4bb-4ebe-8b5e-5dd7e3c2fe09/prometheus-operator-admission-webhook/0.log" Jun 02 03:05:17.472959 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.472877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/thanos-query/0.log" Jun 02 03:05:17.494988 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.494961 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/kube-rbac-proxy-web/0.log" Jun 02 03:05:17.517961 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.517918 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/kube-rbac-proxy/0.log" Jun 02 03:05:17.568432 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.568406 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/prom-label-proxy/0.log" Jun 02 03:05:17.592729 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.592702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/kube-rbac-proxy-rules/0.log" Jun 02 03:05:17.619706 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:17.619630 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59c44fb4b9-s5hkq_91f27e28-4a03-4b05-bb6d-ecd8c7e6dd4e/kube-rbac-proxy-metrics/0.log" Jun 02 03:05:18.789133 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.789102 2563 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws"] Jun 02 03:05:18.797189 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.797162 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:18.809366 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.809334 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws"] Jun 02 03:05:18.970337 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.970301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-proc\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:18.970516 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.970348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-lib-modules\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:18.970516 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.970379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2p5\" (UniqueName: \"kubernetes.io/projected/cf17666d-dbbc-4b00-ae91-2563276069f4-kube-api-access-kl2p5\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:18.970516 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.970442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-sys\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:18.970732 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:18.970537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-podres\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071442 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071371 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-sys\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071584 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071441 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-podres\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071584 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071492 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-sys\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071584 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071535 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-proc\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071584 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071560 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-lib-modules\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071714 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071610 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-podres\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071714 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071629 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-proc\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071714 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071616 2563 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2p5\" (UniqueName: \"kubernetes.io/projected/cf17666d-dbbc-4b00-ae91-2563276069f4-kube-api-access-kl2p5\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.071714 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.071674 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf17666d-dbbc-4b00-ae91-2563276069f4-lib-modules\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.089114 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.089068 2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2p5\" (UniqueName: \"kubernetes.io/projected/cf17666d-dbbc-4b00-ae91-2563276069f4-kube-api-access-kl2p5\") pod \"perf-node-gather-daemonset-d8jws\" (UID: \"cf17666d-dbbc-4b00-ae91-2563276069f4\") " pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.117546 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.117511 2563 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.313881 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.313851 2563 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws"] Jun 02 03:05:19.318949 ip-10-0-138-19 kubenswrapper[2563]: W0602 03:05:19.318914 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcf17666d_dbbc_4b00_ae91_2563276069f4.slice/crio-207c15789751b23289c3ab95784a2dd9abdfa51b233826acdfc4c89c610afc5d WatchSource:0}: Error finding container 207c15789751b23289c3ab95784a2dd9abdfa51b233826acdfc4c89c610afc5d: Status 404 returned error can't find the container with id 207c15789751b23289c3ab95784a2dd9abdfa51b233826acdfc4c89c610afc5d Jun 02 03:05:19.383285 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.383229 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 03:05:19.391551 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.391528 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/2.log" Jun 02 03:05:19.922757 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.922727 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" event={"ID":"cf17666d-dbbc-4b00-ae91-2563276069f4","Type":"ContainerStarted","Data":"85f1fd4d17c47d2c3c05bf4af0d5c8b8a9555a4d89eea84348b7235e4564540b"} Jun 02 03:05:19.922757 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.922763 2563 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" event={"ID":"cf17666d-dbbc-4b00-ae91-2563276069f4","Type":"ContainerStarted","Data":"207c15789751b23289c3ab95784a2dd9abdfa51b233826acdfc4c89c610afc5d"} Jun 02 03:05:19.923257 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.922874 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:19.954531 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.954463 2563 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" podStartSLOduration=1.954444155 podStartE2EDuration="1.954444155s" podCreationTimestamp="2026-06-02 03:05:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 03:05:19.95159834 +0000 UTC m=+1199.923198824" watchObservedRunningTime="2026-06-02 03:05:19.954444155 +0000 UTC m=+1199.926044642" Jun 02 03:05:19.967993 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:19.967964 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75ccdbdd7b-6jhjn_3b92604a-e661-4ba7-a5aa-28e7079c5be2/console/0.log" Jun 02 03:05:20.111447 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:20.111404 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-lx7qk_dd45154e-530c-4be5-9cca-37172cf641f6/download-server/0.log" Jun 02 03:05:20.500466 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:20.500058 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 03:05:20.500466 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:20.500300 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-wsz4b_9f97f813-3603-4f4f-884e-5a4492e8e0ef/console-operator/1.log" Jun 02 03:05:20.502998 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:20.502979 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 03:05:20.503558 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:20.503536 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hd4s2_513f443e-a2f6-401d-a082-4222d02fdcd0/ovn-acl-logging/0.log" Jun 02 03:05:21.572333 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:21.572303 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2m228_6270d4e2-89a3-42a9-8b55-483bfb2f8f9a/dns/0.log" Jun 02 03:05:21.598414 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:21.598392 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2m228_6270d4e2-89a3-42a9-8b55-483bfb2f8f9a/kube-rbac-proxy/0.log" Jun 02 03:05:21.743512 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:21.743482 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hwplc_bc76ee11-6d2c-4cf2-ac0f-b8f35ba0865c/dns-node-resolver/0.log" Jun 02 03:05:22.266398 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:22.266373 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gd2qn_cb382c8c-5050-49a7-a71f-75325166bbc9/node-ca/0.log" Jun 02 03:05:23.159037 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:23.159010 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bmwbw6_995f0e50-2d56-4e89-a047-36ddd51dd2b7/istio-proxy/0.log" Jun 02 03:05:23.318284 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:23.318253 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-69595_4d3ee16b-779a-4fac-83df-9ed85c9763fb/istio-proxy/0.log" Jun 02 03:05:23.369223 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:23.369191 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-9gzvs_0a219430-e140-48e4-9e71-d06c92f6f277/payload-processing/0.log" Jun 02 03:05:23.979207 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:23.979175 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q6mst_4ddc9c4c-0641-48c1-a4d6-ccb64393c120/serve-healthcheck-canary/0.log" Jun 02 03:05:24.454210 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:24.454179 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-dt6tw_ec240069-31bf-4333-b5c0-90829b8c73f0/insights-operator/0.log" Jun 02 03:05:24.454929 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:24.454909 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-788bf58df5-dt6tw_ec240069-31bf-4333-b5c0-90829b8c73f0/insights-operator/1.log" Jun 02 03:05:24.617963 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:24.617938 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqk28_42649385-dcf3-44aa-8911-0232b0e26ac5/kube-rbac-proxy/0.log" Jun 02 03:05:24.639940 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:24.639911 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqk28_42649385-dcf3-44aa-8911-0232b0e26ac5/exporter/0.log" Jun 02 03:05:24.662912 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:24.662892 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zqk28_42649385-dcf3-44aa-8911-0232b0e26ac5/extractor/0.log" Jun 02 03:05:25.939131 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:25.939090 2563 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sfk5k/perf-node-gather-daemonset-d8jws" Jun 02 03:05:26.731953 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:26.731928 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6d9fb5d8dc-zqvf9_682ca309-f070-4a72-ba11-aa63638a6764/nginx/0.log" Jun 02 03:05:27.303882 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:27.303855 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-wdfhw_c3b48e53-1f3c-4143-8b10-766db3ea33fc/manager/0.log" Jun 02 03:05:27.350118 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:27.350095 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f8cc9fd78-wdfhw_c3b48e53-1f3c-4143-8b10-766db3ea33fc/copy-manifests/0.log" Jun 02 03:05:29.160515 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:29.160487 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-774f9bf4f6-kksl2_b881c2f6-e45d-4d9d-8a65-70cc6d1e1d4d/manager/0.log" Jun 02 03:05:29.207139 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:29.207114 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-55h4d_fbaabd11-9a12-496b-921b-199d13c576c9/openshift-lws-operator/0.log" Jun 02 03:05:34.173651 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:34.173627 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-lspf2_25221423-6136-4101-ba83-4345e9fc0ea7/migrator/0.log" Jun 02 03:05:34.196193 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:34.196169 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-lspf2_25221423-6136-4101-ba83-4345e9fc0ea7/graceful-termination/0.log" Jun 02 03:05:34.587187 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:34.587068 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2rd92_6f60493a-6291-4ded-971b-8c016d0fc23e/kube-storage-version-migrator-operator/1.log" Jun 02 03:05:34.588895 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:34.588873 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2rd92_6f60493a-6291-4ded-971b-8c016d0fc23e/kube-storage-version-migrator-operator/0.log" Jun 02 03:05:35.828266 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.828134 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/kube-multus-additional-cni-plugins/0.log" Jun 02 03:05:35.851916 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.851884 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/egress-router-binary-copy/0.log" Jun 02 03:05:35.875238 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.875208 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/cni-plugins/0.log" Jun 02 03:05:35.897472 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.897445 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/bond-cni-plugin/0.log" Jun 02 03:05:35.920541 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.920488 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/routeoverride-cni/0.log" Jun 02 03:05:35.949498 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.949462 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/whereabouts-cni-bincopy/0.log" Jun 02 03:05:35.979119 ip-10-0-138-19 kubenswrapper[2563]: I0602 03:05:35.979086 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fhbxx_0110ac79-5ae3-4da0-9700-60cd268dc3a9/whereabouts-cni/0.log"