Jun 11 16:44:04.000858 ip-10-0-128-197 systemd[1]: Starting Kubernetes Kubelet... Jun 11 16:44:04.362213 ip-10-0-128-197 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:44:04.362213 ip-10-0-128-197 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 16:44:04.362213 ip-10-0-128-197 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:44:04.362213 ip-10-0-128-197 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 16:44:04.362853 ip-10-0-128-197 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:44:04.363419 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.363334 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 16:44:04.367524 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367509 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.367524 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367525 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367530 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367535 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367538 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367541 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367544 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367548 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367551 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367554 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367557 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367560 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367563 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367566 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367569 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367572 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367588 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367591 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367594 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367597 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.367608 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367599 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367606 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367608 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367611 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367614 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367617 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367620 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367623 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367625 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367628 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367630 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367633 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367635 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367638 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367640 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367643 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367645 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367648 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367650 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367652 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.368058 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367655 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367658 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367660 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367663 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367665 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367669 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367673 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367676 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367678 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367681 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367684 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367686 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367688 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367691 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367695 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367698 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367701 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367703 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367705 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367709 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.368734 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367711 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367714 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367717 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367719 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367722 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367724 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367728 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367730 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367733 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367735 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367738 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367740 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367743 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367745 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367748 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367750 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367753 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367755 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367758 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367760 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.369558 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367764 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367768 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367770 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367773 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367777 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367780 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367783 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367787 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367789 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367792 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367796 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367798 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367801 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367804 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367807 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367810 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367812 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367815 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.367818 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.370251 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370077 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370092 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370097 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370100 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370103 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370106 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370109 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370112 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370115 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370117 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370120 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370123 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370126 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370129 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370131 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370134 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370137 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370139 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370142 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370146 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.370779 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370148 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370151 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370153 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370156 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370159 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370161 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370163 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370166 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370169 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370171 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370174 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370176 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370179 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370182 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370186 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370189 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370191 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370194 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370196 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370199 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.371298 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370202 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370204 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370207 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370209 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370213 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370218 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370221 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370224 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370227 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370230 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370235 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370237 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370240 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370242 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370245 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370247 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370250 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370253 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370257 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.371839 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370261 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370263 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370266 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370269 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370271 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370274 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370277 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370280 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370284 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370287 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370289 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370292 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370294 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370297 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370300 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370302 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370305 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370308 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370310 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370313 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.372309 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370315 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370318 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370322 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370324 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370327 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370330 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370333 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370335 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370339 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370343 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370346 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370349 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370352 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370355 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370358 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370360 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370363 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370366 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370368 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.372821 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370371 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370759 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370764 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370767 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370770 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370773 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370776 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370780 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370782 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370785 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370788 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370791 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370794 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370796 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370799 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370802 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370805 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370808 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370810 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370813 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.373266 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370816 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370819 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370822 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370824 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370827 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370829 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370832 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370834 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370837 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370840 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370843 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370845 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370848 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370850 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370854 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370857 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370860 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370863 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370865 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370868 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.373785 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370871 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370873 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370876 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370879 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370881 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370884 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370886 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370889 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370891 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370894 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370897 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370899 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370902 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370905 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370908 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370910 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370912 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370916 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370920 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370923 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.374263 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370926 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370929 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370932 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370934 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370937 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370939 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370942 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370944 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370947 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370949 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370952 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370955 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370957 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370960 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370963 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370966 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370968 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370971 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370973 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370976 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.374749 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370978 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370981 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370983 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370986 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370989 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370992 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370995 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.370998 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371001 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371004 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371006 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371009 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371015 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371018 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371020 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371023 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371025 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371027 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371031 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.375223 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371034 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371103 2557 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371110 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371116 2557 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371121 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371125 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371128 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371132 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371136 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371140 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371143 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371146 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371150 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371153 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371156 2557 flags.go:64] FLAG: --cgroup-root="" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371159 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371162 2557 flags.go:64] FLAG: --client-ca-file="" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371165 2557 flags.go:64] FLAG: --cloud-provider="external" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371168 2557 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371173 2557 flags.go:64] FLAG: --cluster-domain="" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371175 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371178 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371181 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371185 2557 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 16:44:04.375750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371189 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371193 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371197 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371200 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371203 2557 flags.go:64] FLAG: --contention-profiling="false" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371205 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371208 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371214 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371216 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371220 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371223 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371226 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371229 2557 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371232 2557 flags.go:64] FLAG: --enable-server="true" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371235 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371240 2557 flags.go:64] FLAG: --event-burst="100" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371243 2557 flags.go:64] FLAG: --event-qps="50" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371246 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371249 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371252 2557 flags.go:64] FLAG: --eviction-hard="" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371255 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371258 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371261 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371264 2557 flags.go:64] FLAG: --eviction-soft="" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371267 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 16:44:04.376347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371270 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371273 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371276 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371279 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371282 2557 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371285 2557 flags.go:64] FLAG: --feature-gates="" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371288 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371291 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371295 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371299 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371302 2557 flags.go:64] FLAG: --healthz-port="10248" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371304 2557 flags.go:64] FLAG: --help="false" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371307 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371311 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371315 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371318 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371321 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371324 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371327 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371330 2557 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371333 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371336 2557 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371338 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371342 2557 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 16:44:04.376991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371344 2557 flags.go:64] FLAG: --kube-reserved="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371347 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371350 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371353 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371356 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371358 2557 flags.go:64] FLAG: --lock-file="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371361 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371364 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371367 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371373 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371376 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371378 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371381 2557 flags.go:64] FLAG: --logging-format="text" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371384 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371387 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371390 2557 flags.go:64] FLAG: --manifest-url="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371394 2557 flags.go:64] FLAG: --manifest-url-header="" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371398 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371401 2557 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371405 2557 flags.go:64] FLAG: --max-pods="110" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371409 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371411 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371416 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371419 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371422 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 16:44:04.377605 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371425 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371428 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371435 2557 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371438 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371441 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371444 2557 flags.go:64] FLAG: --pod-cidr="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371447 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371452 2557 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371455 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371458 2557 flags.go:64] FLAG: --pods-per-core="0" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371461 2557 flags.go:64] FLAG: --port="10250" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371464 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371467 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0078317a0a1681156" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371470 2557 flags.go:64] FLAG: --qos-reserved="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371472 2557 flags.go:64] FLAG: --read-only-port="10255" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371476 2557 flags.go:64] FLAG: --register-node="true" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371479 2557 flags.go:64] FLAG: --register-with-taints="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371482 2557 flags.go:64] FLAG: --registry-burst="10" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371485 2557 flags.go:64] FLAG: --registry-qps="5" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371488 2557 flags.go:64] FLAG: --reserved-cpus="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371491 2557 flags.go:64] FLAG: --reserved-memory="" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371494 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371497 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371500 2557 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371505 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 16:44:04.378232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371507 2557 flags.go:64] FLAG: --runonce="false" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371510 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371513 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371516 2557 flags.go:64] FLAG: --seccomp-default="false" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371520 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371522 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371526 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371529 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371532 2557 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371534 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371537 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371540 2557 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371542 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371545 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371548 2557 flags.go:64] FLAG: --system-cgroups="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371551 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371556 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371558 2557 flags.go:64] FLAG: --tls-cert-file="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371561 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371565 2557 flags.go:64] FLAG: --tls-min-version="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371568 2557 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371570 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371573 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371591 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371594 2557 flags.go:64] FLAG: --v="2" Jun 11 16:44:04.378888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371598 2557 flags.go:64] FLAG: --version="false" Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371603 2557 flags.go:64] FLAG: --vmodule="" Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371607 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.371610 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371731 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371735 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371739 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371742 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371744 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371747 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371750 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371754 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371757 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371759 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371762 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371765 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371767 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371770 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371772 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371775 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371777 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.379486 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371780 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371782 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371785 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371787 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371791 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371795 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371798 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371800 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371803 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371806 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371808 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371811 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371814 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371817 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371819 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371822 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371825 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371829 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371832 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.380064 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371835 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371837 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371840 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371847 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371851 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371854 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371857 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371860 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371863 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371866 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371868 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371871 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371874 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371877 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371879 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371882 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371885 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371887 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371890 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371892 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.380533 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371895 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371897 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371900 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371903 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371906 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371908 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371911 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371914 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371916 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371919 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371923 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371925 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371928 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371930 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371933 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371936 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371939 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371941 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371944 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.381039 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371947 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371949 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371952 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371955 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371957 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371960 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371962 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371965 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371967 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371971 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371975 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371978 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371980 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371983 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371985 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371988 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371990 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371993 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371996 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.371998 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.381640 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.372001 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.372004 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.372006 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.372010 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.372589 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.379379 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.379396 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.380164 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.380172 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380232 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380237 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380241 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380245 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380249 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.382129 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380252 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380255 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380258 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380261 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380265 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380269 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380272 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380275 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380278 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380280 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380283 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380286 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380288 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380291 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380294 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380297 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380300 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380303 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380306 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.382493 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380309 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380313 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380317 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380319 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380322 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380325 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380328 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380330 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380333 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380335 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380338 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380341 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380344 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380347 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380349 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380352 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380355 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380357 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380360 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.382981 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380362 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380365 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380368 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380370 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380373 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380375 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380378 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380380 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380383 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380386 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380389 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380392 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380395 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380397 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380401 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380403 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380406 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380409 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380411 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380413 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.383447 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380416 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380419 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380422 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380424 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380427 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380430 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380433 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380435 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380438 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380440 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380443 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380445 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380448 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380451 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380453 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380456 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380458 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380461 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380463 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.384025 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380466 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380469 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380471 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380474 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380477 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380480 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380482 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380486 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380489 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380491 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380494 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380497 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380499 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380502 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380504 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380506 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.384480 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380509 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.380514 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380643 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380649 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380651 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380656 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380659 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380663 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380666 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380669 2557 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380671 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380674 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380677 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380679 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380682 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:44:04.384875 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380684 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380687 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380690 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380692 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380695 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380697 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380700 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380703 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380706 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380709 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380712 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380715 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380718 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380720 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380723 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380726 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380728 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380731 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380734 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380736 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:44:04.385240 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380739 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380742 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380744 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380747 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380750 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380753 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380756 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380758 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380761 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380763 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380766 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380768 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380771 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380774 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380776 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380779 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380781 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380783 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380786 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:44:04.385752 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380789 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380792 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380794 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380797 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380800 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380803 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380805 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380808 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380810 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380813 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380816 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380819 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380822 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380825 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380828 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380830 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380833 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380836 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380838 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380841 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:44:04.386208 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380844 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380846 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380849 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380852 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380854 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380857 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380859 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380862 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380864 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380867 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380869 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380873 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380876 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380879 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380881 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380884 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380886 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380890 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380893 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380895 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:44:04.386703 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380898 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380900 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380904 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380908 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380910 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380913 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:04.380916 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.380921 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.381042 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.382960 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.384148 2557 server.go:1019] "Starting client certificate rotation" Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.384253 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:44:04.387207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.385378 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:44:04.387610 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.387595 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:44:04.390219 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.390199 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:44:04.402779 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.402758 2557 log.go:25] "Validated CRI v1 runtime API" Jun 11 16:44:04.405563 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.405545 2557 log.go:25] "Validated CRI v1 image API" Jun 11 16:44:04.406339 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.406326 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 16:44:04.408841 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.408822 2557 fs.go:135] Filesystem UUIDs: map[6790cbcb-07dd-4e8b-8188-102e9472adba:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7ce28ec9-befe-4baa-9b58-af0fc58db1e1:/dev/nvme0n1p4] Jun 11 16:44:04.408902 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.408840 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 16:44:04.413843 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.413723 2557 manager.go:216] Machine: {Timestamp:2026-06-11 16:44:04.412517214 +0000 UTC m=+0.301648581 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100786 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26750d6424898adb3dc5738020be3a SystemUUID:ec26750d-6424-898a-db3d-c5738020be3a BootID:cc7ab6df-84d8-43e2-acc4-59440fd21f94 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:59:b4:09:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:59:b4:09:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:fb:8d:aa:c4:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 16:44:04.413843 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.413831 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 16:44:04.413983 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.413916 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 16:44:04.414912 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.414893 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:44:04.417190 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.417153 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 16:44:04.417358 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.417195 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-197.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 16:44:04.418443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.418431 2557 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 16:44:04.418508 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.418449 2557 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 16:44:04.418508 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.418460 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:44:04.419213 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.419201 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:44:04.420050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.420039 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 16:44:04.423018 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423003 2557 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:44:04.423340 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423327 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 16:44:04.423405 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423369 2557 kubelet.go:486] "Attempting to sync node with API server" Jun 11 16:44:04.423405 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423384 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 16:44:04.423501 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423406 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 16:44:04.423501 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423420 2557 kubelet.go:398] "Adding apiserver pod source" Jun 11 16:44:04.423501 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423433 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 16:44:04.423839 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423825 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:44:04.423899 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.423850 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:44:04.425881 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.425865 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 16:44:04.427204 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.427191 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 16:44:04.427258 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.427221 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 16:44:04.428140 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428129 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 16:44:04.428174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428145 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 16:44:04.428174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428152 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 16:44:04.428174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428157 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 16:44:04.428174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428167 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428180 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428189 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428194 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428201 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428206 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428223 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 16:44:04.428290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.428229 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 16:44:04.429869 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.429856 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 16:44:04.429869 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.429870 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 16:44:04.433241 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.433226 2557 server.go:1268] "Started kubelet" Jun 11 16:44:04.433333 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.433299 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 16:44:04.433480 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.433301 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 16:44:04.433600 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.433516 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 16:44:04.434263 ip-10-0-128-197 systemd[1]: Started Kubernetes Kubelet. Jun 11 16:44:04.434846 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.434811 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-197.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 16:44:04.434846 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.434814 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 16:44:04.435212 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.435192 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 16:44:04.435571 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.435559 2557 server.go:310] "Adding debug handlers to kubelet server" Jun 11 16:44:04.435571 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.435567 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 16:44:04.440937 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.440905 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 16:44:04.443128 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.442232 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18b8148c10b3515d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-06-11 16:44:04.433195357 +0000 UTC m=+0.322326725,LastTimestamp:2026-06-11 16:44:04.433195357 +0000 UTC m=+0.322326725,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}" Jun 11 16:44:04.445101 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445082 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 16:44:04.445203 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.445184 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.445203 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445082 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 16:44:04.445625 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445609 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 16:44:04.445775 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445757 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 16:44:04.445775 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445627 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 16:44:04.445892 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445844 2557 reconstruct.go:97] "Volume reconstruction finished" Jun 11 16:44:04.445892 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.445852 2557 reconciler.go:29] "Reconciler: start to sync state" Jun 11 16:44:04.446133 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446114 2557 factory.go:55] Registering systemd factory Jun 11 16:44:04.446133 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446132 2557 factory.go:223] Registration of the systemd container factory successfully Jun 11 16:44:04.446421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446399 2557 factory.go:153] Registering CRI-O factory Jun 11 16:44:04.446421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446422 2557 factory.go:223] Registration of the crio container factory successfully Jun 11 16:44:04.446553 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446476 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 16:44:04.446553 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446502 2557 factory.go:103] Registering Raw factory Jun 11 16:44:04.446553 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446517 2557 manager.go:1196] Started watching for new ooms in manager Jun 11 16:44:04.446940 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.446923 2557 manager.go:349] Starting recovery of all containers Jun 11 16:44:04.448892 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.448862 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 16:44:04.449029 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.449005 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 16:44:04.454050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.453893 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 16:44:04.456284 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.456269 2557 manager.go:354] Recovery completed Jun 11 16:44:04.461078 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.461065 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.462933 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.462916 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.463014 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.462948 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.463014 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.462959 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.463398 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.463383 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 16:44:04.463398 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.463397 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 16:44:04.463517 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.463412 2557 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:44:04.463517 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.463449 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.464281 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464268 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.464326 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464293 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.464326 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464306 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.464871 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464857 2557 policy_none.go:49] "None policy: Start" Jun 11 16:44:04.464911 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464876 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 16:44:04.464911 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.464896 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 16:44:04.464963 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.464852 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18b8148c127911a4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-06-11 16:44:04.462932388 +0000 UTC m=+0.352063765,LastTimestamp:2026-06-11 16:44:04.462932388 +0000 UTC m=+0.352063765,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}" Jun 11 16:44:04.466007 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.465995 2557 policy_none.go:47] "Start" Jun 11 16:44:04.467850 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.467832 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c54n4" Jun 11 16:44:04.474071 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.473998 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18b8148c12796462 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-06-11 16:44:04.46295357 +0000 UTC m=+0.352084938,LastTimestamp:2026-06-11 16:44:04.46295357 +0000 UTC m=+0.352084938,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}" Jun 11 16:44:04.476508 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.476445 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18b8148c127986f6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-06-11 16:44:04.462962422 +0000 UTC m=+0.352093790,LastTimestamp:2026-06-11 16:44:04.462962422 +0000 UTC m=+0.352093790,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}" Jun 11 16:44:04.478441 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.478426 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c54n4" Jun 11 16:44:04.503245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503230 2557 manager.go:337] "Starting Device Plugin manager" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.503264 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503277 2557 server.go:87] "Starting device plugin registration server" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503520 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503530 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503636 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503757 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.503767 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.504359 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.504410 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.511205 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.511239 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.511259 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.511286 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 16:44:04.513752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.513334 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:44:04.604670 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.604646 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.605802 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.605784 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.605894 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.605814 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.605894 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.605825 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.605894 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.605849 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.611792 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.611771 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"] Jun 11 16:44:04.611845 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.611836 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.613715 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.613669 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.613715 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.613697 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.613715 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.613707 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.615043 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.615032 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.615166 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.615151 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.615203 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.615184 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.615747 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.615734 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.615783 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.615753 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-197.ec2.internal\": node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.616721 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616705 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.616783 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616737 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.616783 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616748 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.616842 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616705 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.616842 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616806 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.616842 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.616819 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.618250 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.618238 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.618293 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.618262 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:44:04.618974 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.618958 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:44:04.619049 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.618984 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:44:04.619049 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.618996 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:44:04.643375 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.643356 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.646601 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.646568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.646659 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.646614 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.646659 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.646639 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ffe0ae6d94a8ca6aa9430e1dfacd020-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"0ffe0ae6d94a8ca6aa9430e1dfacd020\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.648297 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.648279 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-197.ec2.internal\" not found" node="ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.652891 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.652873 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-197.ec2.internal\" not found" node="ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.743959 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.743921 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.747211 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747190 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.747289 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747219 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.747289 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ffe0ae6d94a8ca6aa9430e1dfacd020-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"0ffe0ae6d94a8ca6aa9430e1dfacd020\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.747359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747298 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.747359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747353 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bd509fbdf56165217bbcc1659f2c615-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"2bd509fbdf56165217bbcc1659f2c615\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.747418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.747385 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ffe0ae6d94a8ca6aa9430e1dfacd020-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"0ffe0ae6d94a8ca6aa9430e1dfacd020\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.844409 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.844362 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.945165 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:04.945084 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:04.952452 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.952432 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:04.956393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:04.956278 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:05.045905 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.045869 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:05.146240 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.146207 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:05.246784 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.246696 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:05.347124 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.347099 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found" Jun 11 16:44:05.383322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.383296 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:44:05.384990 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.384968 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 16:44:05.385112 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.385084 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 16:44:05.424293 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.424268 2557 apiserver.go:52] "Watching apiserver" Jun 11 16:44:05.442759 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.442731 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 16:44:05.445411 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.445392 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 16:44:05.445509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.445412 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" Jun 11 16:44:05.445509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.445411 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tvwqv","kube-system/global-pull-secret-syncer-7sddr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9","openshift-cluster-node-tuning-operator/tuned-h4wpg","openshift-dns/node-resolver-k2nzc","openshift-multus/multus-777sc","openshift-multus/network-metrics-daemon-k8t85","openshift-ovn-kubernetes/ovnkube-node-s8cmj","kube-system/konnectivity-agent-zg2zm","openshift-image-registry/node-ca-bjlhc","openshift-multus/multus-additional-cni-plugins-fxs4v","openshift-network-diagnostics/network-check-target-gncdw"] Jun 11 16:44:05.447892 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.447868 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.449900 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.449862 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.450018 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.449996 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:05.450082 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450064 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.450136 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450078 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 16:44:05.450136 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450108 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xvcpc\"" Jun 11 16:44:05.450267 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450252 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.450330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450312 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8cbb86c3-f84d-4721-9e54-1665c93a761c-iptables-alerter-script\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.450365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450345 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cbb86c3-f84d-4721-9e54-1665c93a761c-host-slash\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.450400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.450368 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqj4\" (UniqueName: \"kubernetes.io/projected/8cbb86c3-f84d-4721-9e54-1665c93a761c-kube-api-access-9pqj4\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.452227 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.452212 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.453403 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.453383 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:44:05.453516 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.453451 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" Jun 11 16:44:05.455393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.455055 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.455393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.455137 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.455393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.455220 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 16:44:05.455568 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.455451 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.455831 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.455800 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-28fk9\"" Jun 11 16:44:05.457972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.457778 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.457972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.457790 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.457972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.457850 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.458118 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.458023 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fv426\"" Jun 11 16:44:05.458803 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.458786 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:44:05.459909 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.459893 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.460005 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.459916 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zgcff\"" Jun 11 16:44:05.460067 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.460050 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-777sc" Jun 11 16:44:05.460400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.460386 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.462217 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.462205 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.462283 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.462256 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:05.463316 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.463296 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 16:44:05.463399 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.463350 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.463399 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.463375 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 16:44:05.463519 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.463435 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lqxbq\"" Jun 11 16:44:05.463631 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.463533 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.464741 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.464725 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.465868 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.465844 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:44:05.467162 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.467674 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467648 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.467746 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467679 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 16:44:05.467746 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467690 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 16:44:05.467865 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467750 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 16:44:05.467865 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467825 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6v5gc\"" Jun 11 16:44:05.467985 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467972 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 16:44:05.467985 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.467982 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.469187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.469174 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 16:44:05.469295 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.469280 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xm2g9\"" Jun 11 16:44:05.469483 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.469469 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 16:44:05.469654 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.469639 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.472213 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.472197 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.472727 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.472706 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 16:44:05.472727 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.472720 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p9dhk\"" Jun 11 16:44:05.472869 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.472759 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 16:44:05.472869 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.472762 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 16:44:05.473747 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.473724 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffe0ae6d94a8ca6aa9430e1dfacd020.slice/crio-ba2c5c5fe7701baead7859b7b1257276a9a21c17f50c40378a9c186db39620c7 WatchSource:0}: Error finding container ba2c5c5fe7701baead7859b7b1257276a9a21c17f50c40378a9c186db39620c7: Status 404 returned error can't find the container with id ba2c5c5fe7701baead7859b7b1257276a9a21c17f50c40378a9c186db39620c7 Jun 11 16:44:05.473967 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.473948 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd509fbdf56165217bbcc1659f2c615.slice/crio-340cdf1a9b822add61d1af601232f08da51067e98b33aa731763d23e4efcbdc7 WatchSource:0}: Error finding container 340cdf1a9b822add61d1af601232f08da51067e98b33aa731763d23e4efcbdc7: Status 404 returned error can't find the container with id 340cdf1a9b822add61d1af601232f08da51067e98b33aa731763d23e4efcbdc7 Jun 11 16:44:05.474204 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474189 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 16:44:05.474249 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474212 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 16:44:05.474386 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474368 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"] Jun 11 16:44:05.474475 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474400 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"] Jun 11 16:44:05.474522 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:05.474570 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.474551 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:05.474742 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.474724 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9f2wc\"" Jun 11 16:44:05.478542 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.478526 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:44:05.481244 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.481211 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 16:39:04 +0000 UTC" deadline="2028-02-25 20:19:41.725866134 +0000 UTC" Jun 11 16:44:05.481320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.481246 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14979h35m36.244624952s" Jun 11 16:44:05.483791 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.483774 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnhwm" Jun 11 16:44:05.496154 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.496138 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnhwm" Jun 11 16:44:05.514435 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.514389 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" event={"ID":"0ffe0ae6d94a8ca6aa9430e1dfacd020","Type":"ContainerStarted","Data":"ba2c5c5fe7701baead7859b7b1257276a9a21c17f50c40378a9c186db39620c7"} Jun 11 16:44:05.515629 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.515606 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"2bd509fbdf56165217bbcc1659f2c615","Type":"ContainerStarted","Data":"340cdf1a9b822add61d1af601232f08da51067e98b33aa731763d23e4efcbdc7"} Jun 11 16:44:05.548629 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.547335 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 16:44:05.550955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.550934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-system-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551054 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.550960 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-bin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551054 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.550975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-hostroot\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551054 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.550988 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-conf-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551054 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551006 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-os-release\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-systemd\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-run\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c34c-6597-4aef-9417-1aa198593e6f-tmp-dir\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-kubelet\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551202 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-etc-kubernetes\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxd5q\" (UniqueName: \"kubernetes.io/projected/23c2f23e-f5f5-4030-89f2-12193f9c1012-kube-api-access-dxd5q\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551235 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-systemd-units\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-etc-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551274 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-ovn\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551304 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6678642-6849-4cda-9772-fcf28a9eb640-host\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-sys\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551340 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-os-release\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-konnectivity-ca\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551374 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-env-overrides\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551390 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-tmp\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551403 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7zl5\" (UniqueName: \"kubernetes.io/projected/4091c34c-6597-4aef-9417-1aa198593e6f-kube-api-access-m7zl5\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551424 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-system-cni-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-kubelet-config\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551452 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-conf\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551470 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqj4\" (UniqueName: \"kubernetes.io/projected/8cbb86c3-f84d-4721-9e54-1665c93a761c-kube-api-access-9pqj4\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551511 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-modprobe-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-netns\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551562 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-cni-binary-copy\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551607 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-agent-certs\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551630 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-netns\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551653 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysconfig\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551676 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-var-lib-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-bin\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551724 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-device-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s5l5\" (UniqueName: \"kubernetes.io/projected/6febe691-cbdf-415d-863e-e836daa57128-kube-api-access-4s5l5\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.551859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-systemd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovn-node-metrics-cert\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551861 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-script-lib\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-etc-tuned\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551889 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551913 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6678642-6849-4cda-9772-fcf28a9eb640-serviceca\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.551972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-var-lib-kubelet\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-socket-dir-parent\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552033 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqzjx\" (UniqueName: \"kubernetes.io/projected/b05df1bd-4184-445d-be31-e2364f0f5079-kube-api-access-pqzjx\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-cnibin\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552118 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-dbus\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.552551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552167 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-lib-modules\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-cnibin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-multus\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552233 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-log-socket\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552257 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552279 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-sys-fs\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552301 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-host\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552324 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-k8s-cni-cncf-io\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8cbb86c3-f84d-4721-9e54-1665c93a761c-iptables-alerter-script\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cbb86c3-f84d-4721-9e54-1665c93a761c-host-slash\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552433 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cbb86c3-f84d-4721-9e54-1665c93a761c-host-slash\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552453 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-daemon-config\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552477 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-slash\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-node-log\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552521 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-config\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552541 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc2pt\" (UniqueName: \"kubernetes.io/projected/6b585434-323b-4b8a-9a66-edb9e17f41e1-kube-api-access-gc2pt\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.553068 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552602 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4091c34c-6597-4aef-9417-1aa198593e6f-hosts-file\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552660 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-multus-certs\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552681 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-kubelet\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552726 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-netd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552750 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552772 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-socket-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552824 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8z4\" (UniqueName: \"kubernetes.io/projected/b544189e-e094-4cf1-b455-90027610aa13-kube-api-access-2n8z4\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552837 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxdls\" (UniqueName: \"kubernetes.io/projected/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-kube-api-access-sxdls\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wt6\" (UniqueName: \"kubernetes.io/projected/d6678642-6849-4cda-9772-fcf28a9eb640-kube-api-access-k9wt6\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552897 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-registration-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-kubernetes\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.553658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.552955 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8cbb86c3-f84d-4721-9e54-1665c93a761c-iptables-alerter-script\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.558004 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.557983 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 16:44:05.560887 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.560871 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqj4\" (UniqueName: \"kubernetes.io/projected/8cbb86c3-f84d-4721-9e54-1665c93a761c-kube-api-access-9pqj4\") pod \"iptables-alerter-tvwqv\" (UID: \"8cbb86c3-f84d-4721-9e54-1665c93a761c\") " pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.653596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653596 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s5l5\" (UniqueName: \"kubernetes.io/projected/6febe691-cbdf-415d-863e-e836daa57128-kube-api-access-4s5l5\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-systemd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovn-node-metrics-cert\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653712 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-systemd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-script-lib\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653728 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653694 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.653766 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653745 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-etc-tuned\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653768 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.653955 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6678642-6849-4cda-9772-fcf28a9eb640-serviceca\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.654033 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-var-lib-kubelet\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.654131 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:06.154086388 +0000 UTC m=+2.043217743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654147 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-var-lib-kubelet\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654194 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-socket-dir-parent\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqzjx\" (UniqueName: \"kubernetes.io/projected/b05df1bd-4184-445d-be31-e2364f0f5079-kube-api-access-pqzjx\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.654245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654244 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-cnibin\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-dbus\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-lib-modules\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-cnibin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-multus\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654405 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-log-socket\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654413 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d6678642-6849-4cda-9772-fcf28a9eb640-serviceca\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-sys-fs\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-host\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654513 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-k8s-cni-cncf-io\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654536 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-lib-modules\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654541 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-script-lib\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-daemon-config\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.654817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654609 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-slash\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654615 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-node-log\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-config\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654665 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-socket-dir-parent\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654672 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-dbus\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654683 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654708 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-cnibin\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654751 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-host\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654752 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-slash\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654790 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-node-log\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654798 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-k8s-cni-cncf-io\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654802 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-cnibin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654822 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654840 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-multus\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654855 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc2pt\" (UniqueName: \"kubernetes.io/projected/6b585434-323b-4b8a-9a66-edb9e17f41e1-kube-api-access-gc2pt\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.655658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654904 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4091c34c-6597-4aef-9417-1aa198593e6f-hosts-file\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-multus-certs\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654878 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-sys-fs\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-kubelet\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.654999 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-multus-certs\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655009 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655009 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-kubelet\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-netd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655063 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655100 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-netd\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655106 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-socket-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655158 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-log-socket\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.655170 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655160 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8z4\" (UniqueName: \"kubernetes.io/projected/b544189e-e094-4cf1-b455-90027610aa13-kube-api-access-2n8z4\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.656464 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.655215 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:06.155200262 +0000 UTC m=+2.044331617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxdls\" (UniqueName: \"kubernetes.io/projected/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-kube-api-access-sxdls\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovnkube-config\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655313 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-daemon-config\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wt6\" (UniqueName: \"kubernetes.io/projected/d6678642-6849-4cda-9772-fcf28a9eb640-kube-api-access-k9wt6\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655353 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-registration-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-kubernetes\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655388 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-system-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655412 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-bin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655432 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-registration-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655441 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-hostroot\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655481 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-hostroot\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655489 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-socket-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655521 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-cni-bin\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655537 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-system-cni-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655558 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-conf-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655560 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-kubernetes\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657344 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655566 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6b585434-323b-4b8a-9a66-edb9e17f41e1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655605 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-multus-conf-dir\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655121 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4091c34c-6597-4aef-9417-1aa198593e6f-hosts-file\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-os-release\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-systemd\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-run\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655700 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-os-release\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c34c-6597-4aef-9417-1aa198593e6f-tmp-dir\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655713 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-systemd\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-kubelet\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655751 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-var-lib-kubelet\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655751 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-run\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-etc-kubernetes\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxd5q\" (UniqueName: \"kubernetes.io/projected/23c2f23e-f5f5-4030-89f2-12193f9c1012-kube-api-access-dxd5q\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655841 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-systemd-units\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655815 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-etc-kubernetes\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-etc-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.657934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655910 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-etc-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655933 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-ovn\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655941 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-systemd-units\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655984 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6678642-6849-4cda-9772-fcf28a9eb640-host\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655987 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-run-ovn\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655953 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655955 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d6678642-6849-4cda-9772-fcf28a9eb640-host\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.655959 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4091c34c-6597-4aef-9417-1aa198593e6f-tmp-dir\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-sys\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656033 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-os-release\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-konnectivity-ca\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-env-overrides\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-tmp\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656105 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-sys\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656120 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-os-release\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7zl5\" (UniqueName: \"kubernetes.io/projected/4091c34c-6597-4aef-9417-1aa198593e6f-kube-api-access-m7zl5\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-system-cni-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656292 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-kubelet-config\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.658465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656343 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b585434-323b-4b8a-9a66-edb9e17f41e1-system-cni-dir\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656390 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-conf\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656417 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1219414a-d398-4f74-a6f2-b59466fbf0da-kubelet-config\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656504 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-env-overrides\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656546 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-modprobe-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656568 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysctl-conf\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656608 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-modprobe-d\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-netns\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656665 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-cni-binary-copy\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-agent-certs\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656712 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-netns\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysconfig\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-var-lib-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656770 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-etc-tuned\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656781 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-bin\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-device-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656807 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-run-netns\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656875 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-host-cni-bin\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656877 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-var-lib-openvswitch\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656927 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b544189e-e094-4cf1-b455-90027610aa13-etc-sysconfig\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.656938 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6febe691-cbdf-415d-863e-e836daa57128-device-dir\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.657134 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-konnectivity-ca\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.657137 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-ovn-node-metrics-cert\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.657182 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23c2f23e-f5f5-4030-89f2-12193f9c1012-cni-binary-copy\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.657225 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23c2f23e-f5f5-4030-89f2-12193f9c1012-host-run-netns\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.658623 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b544189e-e094-4cf1-b455-90027610aa13-tmp\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.659685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.659059 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88-agent-certs\") pod \"konnectivity-agent-zg2zm\" (UID: \"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88\") " pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.662528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.662396 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:05.662528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.662421 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:05.662528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.662434 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:05.662528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:05.662486 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:06.162470629 +0000 UTC m=+2.051601994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:05.663070 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.662681 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s5l5\" (UniqueName: \"kubernetes.io/projected/6febe691-cbdf-415d-863e-e836daa57128-kube-api-access-4s5l5\") pod \"aws-ebs-csi-driver-node-xxxq9\" (UID: \"6febe691-cbdf-415d-863e-e836daa57128\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.664332 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.664307 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqzjx\" (UniqueName: \"kubernetes.io/projected/b05df1bd-4184-445d-be31-e2364f0f5079-kube-api-access-pqzjx\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:05.664696 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.664672 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc2pt\" (UniqueName: \"kubernetes.io/projected/6b585434-323b-4b8a-9a66-edb9e17f41e1-kube-api-access-gc2pt\") pod \"multus-additional-cni-plugins-fxs4v\" (UID: \"6b585434-323b-4b8a-9a66-edb9e17f41e1\") " pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.664799 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.664678 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxdls\" (UniqueName: \"kubernetes.io/projected/4c925d43-c0ac-4c8c-99a2-fd146d4b9644-kube-api-access-sxdls\") pod \"ovnkube-node-s8cmj\" (UID: \"4c925d43-c0ac-4c8c-99a2-fd146d4b9644\") " pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.665018 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.664993 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7zl5\" (UniqueName: \"kubernetes.io/projected/4091c34c-6597-4aef-9417-1aa198593e6f-kube-api-access-m7zl5\") pod \"node-resolver-k2nzc\" (UID: \"4091c34c-6597-4aef-9417-1aa198593e6f\") " pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.665270 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.665246 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wt6\" (UniqueName: \"kubernetes.io/projected/d6678642-6849-4cda-9772-fcf28a9eb640-kube-api-access-k9wt6\") pod \"node-ca-bjlhc\" (UID: \"d6678642-6849-4cda-9772-fcf28a9eb640\") " pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.665648 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.665621 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxd5q\" (UniqueName: \"kubernetes.io/projected/23c2f23e-f5f5-4030-89f2-12193f9c1012-kube-api-access-dxd5q\") pod \"multus-777sc\" (UID: \"23c2f23e-f5f5-4030-89f2-12193f9c1012\") " pod="openshift-multus/multus-777sc" Jun 11 16:44:05.665915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.665900 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8z4\" (UniqueName: \"kubernetes.io/projected/b544189e-e094-4cf1-b455-90027610aa13-kube-api-access-2n8z4\") pod \"tuned-h4wpg\" (UID: \"b544189e-e094-4cf1-b455-90027610aa13\") " pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.766752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.765909 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:44:05.772638 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.772615 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:44:05.776423 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.776409 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tvwqv" Jun 11 16:44:05.782474 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.782451 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" Jun 11 16:44:05.786276 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.786255 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cbb86c3_f84d_4721_9e54_1665c93a761c.slice/crio-2d801cdea55b55c6999070d3892c5089a6c6ea3cd6916fd2fe4b2bc66c1f1518 WatchSource:0}: Error finding container 2d801cdea55b55c6999070d3892c5089a6c6ea3cd6916fd2fe4b2bc66c1f1518: Status 404 returned error can't find the container with id 2d801cdea55b55c6999070d3892c5089a6c6ea3cd6916fd2fe4b2bc66c1f1518 Jun 11 16:44:05.789767 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.789746 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" Jun 11 16:44:05.791442 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.791421 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6febe691_cbdf_415d_863e_e836daa57128.slice/crio-5521c09747f51236fdecea7f0ad4d4385ee72f61311b2e6ce880b0bdc1745af8 WatchSource:0}: Error finding container 5521c09747f51236fdecea7f0ad4d4385ee72f61311b2e6ce880b0bdc1745af8: Status 404 returned error can't find the container with id 5521c09747f51236fdecea7f0ad4d4385ee72f61311b2e6ce880b0bdc1745af8 Jun 11 16:44:05.792528 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.792513 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k2nzc" Jun 11 16:44:05.798086 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.798067 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-777sc" Jun 11 16:44:05.800854 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.800798 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb544189e_e094_4cf1_b455_90027610aa13.slice/crio-448bde95212ba7c6edb88cc8ad0ee38a1a517cd0c7da82ef56264f3c6b5d2f10 WatchSource:0}: Error finding container 448bde95212ba7c6edb88cc8ad0ee38a1a517cd0c7da82ef56264f3c6b5d2f10: Status 404 returned error can't find the container with id 448bde95212ba7c6edb88cc8ad0ee38a1a517cd0c7da82ef56264f3c6b5d2f10 Jun 11 16:44:05.803909 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.803879 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:05.804552 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.804520 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4091c34c_6597_4aef_9417_1aa198593e6f.slice/crio-1462a114e88f7b0ffe069cb70d1efb4632e63cb6c2797689a56bf2da701cd624 WatchSource:0}: Error finding container 1462a114e88f7b0ffe069cb70d1efb4632e63cb6c2797689a56bf2da701cd624: Status 404 returned error can't find the container with id 1462a114e88f7b0ffe069cb70d1efb4632e63cb6c2797689a56bf2da701cd624 Jun 11 16:44:05.808301 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.808278 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c2f23e_f5f5_4030_89f2_12193f9c1012.slice/crio-8edca8ec8adf663f5bb6b93ae63c3e97abf9ba8bd3ca0b5fe33e0b108b6b1ddc WatchSource:0}: Error finding container 8edca8ec8adf663f5bb6b93ae63c3e97abf9ba8bd3ca0b5fe33e0b108b6b1ddc: Status 404 returned error can't find the container with id 8edca8ec8adf663f5bb6b93ae63c3e97abf9ba8bd3ca0b5fe33e0b108b6b1ddc Jun 11 16:44:05.809454 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.809434 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:05.812881 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.812862 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c925d43_c0ac_4c8c_99a2_fd146d4b9644.slice/crio-da0fbeff736e6eebdbae336e0ec87e702ec9ef5933e00dd7d5ebab2e4ce05ed9 WatchSource:0}: Error finding container da0fbeff736e6eebdbae336e0ec87e702ec9ef5933e00dd7d5ebab2e4ce05ed9: Status 404 returned error can't find the container with id da0fbeff736e6eebdbae336e0ec87e702ec9ef5933e00dd7d5ebab2e4ce05ed9 Jun 11 16:44:05.816106 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.816063 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bjlhc" Jun 11 16:44:05.819224 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.819206 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a8fc95c_aaf4_4f9e_bcc7_d6bdb8ae5e88.slice/crio-1912ea35ebbd04fe2f17fc0bb03a4f417622f0dbe42a4d20ad3e4e28dde3cd0a WatchSource:0}: Error finding container 1912ea35ebbd04fe2f17fc0bb03a4f417622f0dbe42a4d20ad3e4e28dde3cd0a: Status 404 returned error can't find the container with id 1912ea35ebbd04fe2f17fc0bb03a4f417622f0dbe42a4d20ad3e4e28dde3cd0a Jun 11 16:44:05.819309 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:05.819276 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" Jun 11 16:44:05.827257 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.827238 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6678642_6849_4cda_9772_fcf28a9eb640.slice/crio-b3aabb258ba64a96793180fd49a54c088280fe361631f8e952aa0ea3a257b5d9 WatchSource:0}: Error finding container b3aabb258ba64a96793180fd49a54c088280fe361631f8e952aa0ea3a257b5d9: Status 404 returned error can't find the container with id b3aabb258ba64a96793180fd49a54c088280fe361631f8e952aa0ea3a257b5d9 Jun 11 16:44:05.829066 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:05.829047 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b585434_323b_4b8a_9a66_edb9e17f41e1.slice/crio-67188c24dee8d02a52d655bcd394a105b16339bc840657a575c791e339bb1d36 WatchSource:0}: Error finding container 67188c24dee8d02a52d655bcd394a105b16339bc840657a575c791e339bb1d36: Status 404 returned error can't find the container with id 67188c24dee8d02a52d655bcd394a105b16339bc840657a575c791e339bb1d36 Jun 11 16:44:06.160459 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.160382 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:06.160459 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.160428 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:06.160697 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.160543 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:06.160697 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.160559 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:06.160697 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.160642 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:07.160617992 +0000 UTC m=+3.049749350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:06.160697 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.160664 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:07.160654087 +0000 UTC m=+3.049785446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:06.262489 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.261806 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:06.262489 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.262023 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:06.262489 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.262044 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:06.262489 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.262058 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:06.262489 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:06.262115 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:07.262097514 +0000 UTC m=+3.151228889 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:06.499233 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.499084 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:39:05 +0000 UTC" deadline="2027-11-09 20:03:32.444287401 +0000 UTC" Jun 11 16:44:06.499233 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.499139 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12387h19m25.945152785s" Jun 11 16:44:06.545862 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.545827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"da0fbeff736e6eebdbae336e0ec87e702ec9ef5933e00dd7d5ebab2e4ce05ed9"} Jun 11 16:44:06.560014 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.559976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-777sc" event={"ID":"23c2f23e-f5f5-4030-89f2-12193f9c1012","Type":"ContainerStarted","Data":"8edca8ec8adf663f5bb6b93ae63c3e97abf9ba8bd3ca0b5fe33e0b108b6b1ddc"} Jun 11 16:44:06.576743 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.576705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k2nzc" event={"ID":"4091c34c-6597-4aef-9417-1aa198593e6f","Type":"ContainerStarted","Data":"1462a114e88f7b0ffe069cb70d1efb4632e63cb6c2797689a56bf2da701cd624"} Jun 11 16:44:06.583058 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.583025 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" event={"ID":"b544189e-e094-4cf1-b455-90027610aa13","Type":"ContainerStarted","Data":"448bde95212ba7c6edb88cc8ad0ee38a1a517cd0c7da82ef56264f3c6b5d2f10"} Jun 11 16:44:06.597646 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.597455 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" event={"ID":"6febe691-cbdf-415d-863e-e836daa57128","Type":"ContainerStarted","Data":"5521c09747f51236fdecea7f0ad4d4385ee72f61311b2e6ce880b0bdc1745af8"} Jun 11 16:44:06.616109 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.616024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerStarted","Data":"67188c24dee8d02a52d655bcd394a105b16339bc840657a575c791e339bb1d36"} Jun 11 16:44:06.626872 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.626790 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bjlhc" event={"ID":"d6678642-6849-4cda-9772-fcf28a9eb640","Type":"ContainerStarted","Data":"b3aabb258ba64a96793180fd49a54c088280fe361631f8e952aa0ea3a257b5d9"} Jun 11 16:44:06.636049 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.635999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zg2zm" event={"ID":"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88","Type":"ContainerStarted","Data":"1912ea35ebbd04fe2f17fc0bb03a4f417622f0dbe42a4d20ad3e4e28dde3cd0a"} Jun 11 16:44:06.645919 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.645886 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvwqv" event={"ID":"8cbb86c3-f84d-4721-9e54-1665c93a761c","Type":"ContainerStarted","Data":"2d801cdea55b55c6999070d3892c5089a6c6ea3cd6916fd2fe4b2bc66c1f1518"} Jun 11 16:44:06.849694 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:06.849612 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:44:07.168849 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.168772 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:07.168849 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.168825 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:07.169070 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.168971 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:07.169070 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.169026 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:09.169008682 +0000 UTC m=+5.058140042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:07.169182 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.169092 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:07.169182 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.169127 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:09.169114003 +0000 UTC m=+5.058245360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:07.270045 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.270012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:07.270202 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.270186 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:07.270278 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.270203 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:07.270278 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.270212 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:07.270278 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.270261 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:09.270243043 +0000 UTC m=+5.159374416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:07.500996 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.500857 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:39:05 +0000 UTC" deadline="2027-12-01 14:52:20.789743661 +0000 UTC" Jun 11 16:44:07.500996 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.500899 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12910h8m13.288849712s" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.513116 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.513240 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.513679 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.513792 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:07.513850 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:07.513955 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:07.513919 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:09.190827 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.190792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:09.191297 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.190853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:09.191297 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.191000 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:09.191297 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.191041 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:09.191297 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.191083 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:13.191064598 +0000 UTC m=+9.080195956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:09.191297 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.191110 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:13.191096936 +0000 UTC m=+9.080228296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:09.292088 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.292051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:09.292724 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.292688 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:09.292724 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.292721 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:09.292922 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.292736 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:09.292922 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.292832 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:13.292810902 +0000 UTC m=+9.181942260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:09.512169 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.512121 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:09.512169 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.512167 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:09.512397 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:09.512127 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:09.512397 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.512257 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:09.512397 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.512338 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:09.512397 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:09.512386 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:11.511632 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:11.511600 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:11.511632 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:11.511628 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:11.511632 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:11.511626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:11.512258 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:11.511730 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:11.512258 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:11.511949 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:11.512258 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:11.512074 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:13.225740 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.225693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:13.226199 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.225763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:13.226199 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.225818 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:13.226199 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.225899 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:21.225878877 +0000 UTC m=+17.115010238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:13.226199 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.225917 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:13.226199 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.225977 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:21.22596105 +0000 UTC m=+17.115092408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:13.326739 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.326703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:13.327017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.326921 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:13.327017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.326942 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:13.327017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.326955 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:13.327017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.327016 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:21.326999679 +0000 UTC m=+17.216131041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:13.511755 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.511609 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:13.511755 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.511657 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:13.511755 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:13.511623 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:13.512030 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.511757 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:13.512030 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.511861 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:13.512030 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:13.511937 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:15.511719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:15.511601 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:15.511719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:15.511651 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:15.511719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:15.511613 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:15.512253 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:15.511733 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:15.512253 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:15.511836 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:15.512253 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:15.511902 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:17.512128 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:17.511896 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:17.512559 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:17.511939 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:17.512559 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:17.512196 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:17.512559 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:17.511939 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:17.512559 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:17.512263 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:17.512559 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:17.512357 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:19.511796 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:19.511752 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:19.512225 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:19.511807 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:19.512225 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:19.511754 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:19.512225 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:19.511898 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:19.512225 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:19.511991 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:19.512225 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:19.512075 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:21.287219 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.287184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:21.287838 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.287234 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:21.287838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.287339 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:21.287838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.287345 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:21.287838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.287405 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:44:37.287389394 +0000 UTC m=+33.176520749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:21.287838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.287424 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:37.287417009 +0000 UTC m=+33.176548363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:21.387942 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.387917 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:21.388104 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.388034 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:21.388104 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.388046 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:21.388104 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.388055 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:21.388104 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.388099 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:37.388086885 +0000 UTC m=+33.277218241 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:21.511479 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.511449 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:21.511669 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.511453 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:21.511669 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.511553 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:21.511669 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.511645 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:21.511669 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:21.511452 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:21.511801 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:21.511711 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:23.512187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:23.512151 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:23.512187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:23.512187 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:23.512187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:23.512196 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:23.512802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:23.512273 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:23.512802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:23.512421 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:23.512802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:23.512539 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:25.512202 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:25.512178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:25.512485 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:25.512219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:25.512485 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:25.512286 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:25.512485 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:25.512302 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:25.512485 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:25.512364 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:25.512485 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:25.512449 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:25.702639 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:25.702572 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" event={"ID":"0ffe0ae6d94a8ca6aa9430e1dfacd020","Type":"ContainerStarted","Data":"18ff80c35c47b54a183ffb35b79c8c60e6d5793fa0e28657fe9a66a4b80b7a83"} Jun 11 16:44:25.717535 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:25.717480 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" podStartSLOduration=20.717458739 podStartE2EDuration="20.717458739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:25.7163483 +0000 UTC m=+21.605479678" watchObservedRunningTime="2026-06-11 16:44:25.717458739 +0000 UTC m=+21.606590118" Jun 11 16:44:26.706883 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.706396 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" event={"ID":"b544189e-e094-4cf1-b455-90027610aa13","Type":"ContainerStarted","Data":"a99ef482a7ad843b98b623cc3af61260fe6f8147841aa1015687a14aee2bfff1"} Jun 11 16:44:26.708088 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.708052 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" event={"ID":"6febe691-cbdf-415d-863e-e836daa57128","Type":"ContainerStarted","Data":"3d66c41b7450458b3c1d9c2e55141d5cf4a7b2d8533507f8790ebfa3bef24dd5"} Jun 11 16:44:26.709479 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.709460 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="513bc3ab30ff9f963521da78616665fd74b29b5c6dd1c39a327de3a968f8d1ce" exitCode=0 Jun 11 16:44:26.709566 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.709524 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"513bc3ab30ff9f963521da78616665fd74b29b5c6dd1c39a327de3a968f8d1ce"} Jun 11 16:44:26.711044 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.710947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bjlhc" event={"ID":"d6678642-6849-4cda-9772-fcf28a9eb640","Type":"ContainerStarted","Data":"44c5c8694d87d5710c44424435d97145d835140ba6b59d0a0d23bf29fe1ad136"} Jun 11 16:44:26.712363 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.712288 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zg2zm" event={"ID":"7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88","Type":"ContainerStarted","Data":"c3a6f69aa17501bad4bd6f01dd6a8b24f3c126051f68e9b61700f0b2e50a5e3c"} Jun 11 16:44:26.713733 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.713711 2557 generic.go:356] "Generic (PLEG): container finished" podID="2bd509fbdf56165217bbcc1659f2c615" containerID="b8b3496444fa8fe51a83c08c73b2028f0a270698c6e42d480b3c59808dc969d2" exitCode=0 Jun 11 16:44:26.713811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.713771 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"2bd509fbdf56165217bbcc1659f2c615","Type":"ContainerDied","Data":"b8b3496444fa8fe51a83c08c73b2028f0a270698c6e42d480b3c59808dc969d2"} Jun 11 16:44:26.716514 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.716497 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:44:26.716850 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.716831 2557 generic.go:356] "Generic (PLEG): container finished" podID="4c925d43-c0ac-4c8c-99a2-fd146d4b9644" containerID="0b77d90e2ca2af964ad49e2ea9521e186e71bbcee36812b8df24ea12d3bb219c" exitCode=1 Jun 11 16:44:26.716949 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.716935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"d10468fc05dbdff1f6ef5bb692d9af50a84ee234aa04abb6256bb0c1fabf987e"} Jun 11 16:44:26.717014 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.717002 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"6478cf0ad80ad566bcaa258c50b047d68c507ea3a06cdac4d5b86e822aa931f0"} Jun 11 16:44:26.717064 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.717038 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"7177235c6217861dcd27bb4ade8ca501cc3424d094eba0bb1307b08361d42d2d"} Jun 11 16:44:26.717111 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.717074 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"c92e22c363ab1d0fb3f40295e036fee0f9597b22dd743f0507ca2c0df31a9fee"} Jun 11 16:44:26.717111 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.717104 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerDied","Data":"0b77d90e2ca2af964ad49e2ea9521e186e71bbcee36812b8df24ea12d3bb219c"} Jun 11 16:44:26.717209 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.717133 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"685f4252f2e141c6187ef9229b2ae68d373061470764db9c5ef992068748acb9"} Jun 11 16:44:26.718341 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.718324 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-777sc" event={"ID":"23c2f23e-f5f5-4030-89f2-12193f9c1012","Type":"ContainerStarted","Data":"04ad376fc94d66e849c8c44efb91030b44af92ec4d793d99538147846bce7faf"} Jun 11 16:44:26.719551 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.719530 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k2nzc" event={"ID":"4091c34c-6597-4aef-9417-1aa198593e6f","Type":"ContainerStarted","Data":"5871003638d522f475fd78429cb5edbb97060d21e3b22e7786ce54c44f7afee6"} Jun 11 16:44:26.727276 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.727242 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h4wpg" podStartSLOduration=3.121260074 podStartE2EDuration="22.727231217s" totalImagesPullingTime="19.605971143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:26.727187576 +0000 UTC m=+22.616318953" watchObservedRunningTime="2026-06-11 16:44:26.727231217 +0000 UTC m=+22.616362628" Jun 11 16:44:26.775236 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.775177 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k2nzc" podStartSLOduration=3.174726689 podStartE2EDuration="22.775155871s" totalImagesPullingTime="19.600429182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:26.774954587 +0000 UTC m=+22.664085967" watchObservedRunningTime="2026-06-11 16:44:26.775155871 +0000 UTC m=+22.664287248" Jun 11 16:44:26.800919 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.800876 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-777sc" podStartSLOduration=2.384111368 podStartE2EDuration="22.800859155s" totalImagesPullingTime="20.416747787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:26.800747054 +0000 UTC m=+22.689878430" watchObservedRunningTime="2026-06-11 16:44:26.800859155 +0000 UTC m=+22.689990533" Jun 11 16:44:26.815870 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.815818 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zg2zm" podStartSLOduration=3.300236153 podStartE2EDuration="22.815796272s" totalImagesPullingTime="19.515560119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:26.815204512 +0000 UTC m=+22.704335883" watchObservedRunningTime="2026-06-11 16:44:26.815796272 +0000 UTC m=+22.704927650" Jun 11 16:44:26.830723 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:26.830685 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bjlhc" podStartSLOduration=3.25305763 podStartE2EDuration="22.830671901s" totalImagesPullingTime="19.577614271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:26.830160866 +0000 UTC m=+22.719292242" watchObservedRunningTime="2026-06-11 16:44:26.830671901 +0000 UTC m=+22.719803277" Jun 11 16:44:27.512192 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.512138 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:27.512192 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.512175 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:27.512433 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.512175 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:27.512433 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:27.512268 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:27.512433 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:27.512373 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:27.512621 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:27.512454 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:27.723880 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.723835 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"2bd509fbdf56165217bbcc1659f2c615","Type":"ContainerStarted","Data":"ed1a7dab1aed078e0aec308f92cf36b5eb50e9f4cb33a028a58465670387ed08"} Jun 11 16:44:27.725555 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.725526 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tvwqv" event={"ID":"8cbb86c3-f84d-4721-9e54-1665c93a761c","Type":"ContainerStarted","Data":"cce7a1193dca06363341240a3a46d72096ff454c7f16b4e9a71a067b1845f01a"} Jun 11 16:44:27.742617 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.742514 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" podStartSLOduration=22.742497483 podStartE2EDuration="22.742497483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:27.742085907 +0000 UTC m=+23.631217286" watchObservedRunningTime="2026-06-11 16:44:27.742497483 +0000 UTC m=+23.631628858" Jun 11 16:44:27.760378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:27.758505 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tvwqv" podStartSLOduration=4.135004375 podStartE2EDuration="23.758487668s" totalImagesPullingTime="19.623483293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:27.758039526 +0000 UTC m=+23.647170903" watchObservedRunningTime="2026-06-11 16:44:27.758487668 +0000 UTC m=+23.647619046" Jun 11 16:44:28.041027 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.041003 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 16:44:28.514081 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.513963 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T16:44:28.041021979Z","UUID":"b18291f2-17e8-44c4-bf7c-5cd38293e70c","Handler":null,"Name":"","Endpoint":""} Jun 11 16:44:28.516085 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.516050 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 16:44:28.516220 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.516100 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 16:44:28.730765 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.730699 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" event={"ID":"6febe691-cbdf-415d-863e-e836daa57128","Type":"ContainerStarted","Data":"ba6f0c311c2a81275a8f4d4cbdeab002fbcc10a395af93c30511ec7633bc93b2"} Jun 11 16:44:28.734758 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.734727 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:44:28.735237 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:28.735210 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"f0c79a9fb0f482d71e92cef4552cdaed5442b5f3ef99742441969dc6cfb0ce44"} Jun 11 16:44:29.512000 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:29.511965 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:29.512253 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:29.511971 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:29.512253 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:29.512088 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:29.512253 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:29.512099 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:29.512253 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:29.512185 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:29.512415 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:29.512283 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:30.829207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:30.829129 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:30.829739 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:30.829722 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:31.512102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.511914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:31.512274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.511929 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:31.512274 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:31.512203 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:31.512274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.511975 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:31.512274 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:31.512254 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:31.512484 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:31.512331 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:31.744163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.744135 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:44:31.744612 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.744560 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"bf6d3ea7940a42c272239f89a948c4d88c8d0b659929625c098ac3a11e1a098d"} Jun 11 16:44:31.744838 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.744819 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:31.745062 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.745037 2557 scope.go:117] "RemoveContainer" containerID="0b77d90e2ca2af964ad49e2ea9521e186e71bbcee36812b8df24ea12d3bb219c" Jun 11 16:44:31.746880 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.746855 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" event={"ID":"6febe691-cbdf-415d-863e-e836daa57128","Type":"ContainerStarted","Data":"51987b9134b5557586d2056be203a773e3596cd1aaa230dcf825deec9ec0ebb6"} Jun 11 16:44:31.748713 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.748693 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="1456c289c46719dc70f2eaa4574b8cfdb70e5ae0a3633b53e6d598097780e5c4" exitCode=0 Jun 11 16:44:31.748805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.748773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"1456c289c46719dc70f2eaa4574b8cfdb70e5ae0a3633b53e6d598097780e5c4"} Jun 11 16:44:31.748981 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.748963 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:31.749660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.749643 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zg2zm" Jun 11 16:44:31.761708 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.761684 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:31.816698 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:31.816480 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxxq9" podStartSLOduration=3.960222623 podStartE2EDuration="27.816464073s" totalImagesPullingTime="23.85624145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:31.802849982 +0000 UTC m=+27.691981360" watchObservedRunningTime="2026-06-11 16:44:31.816464073 +0000 UTC m=+27.705595500" Jun 11 16:44:32.754788 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.754766 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:44:32.755573 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.755083 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" event={"ID":"4c925d43-c0ac-4c8c-99a2-fd146d4b9644","Type":"ContainerStarted","Data":"c8ae5922e8422d1794e2d2bcd65e931e625cba4aef8b0c4c86863eeced10d329"} Jun 11 16:44:32.755573 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.755541 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:32.755731 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.755646 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:32.770611 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.770570 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:44:32.791491 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.791448 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" podStartSLOduration=8.943605051 podStartE2EDuration="28.791435916s" totalImagesPullingTime="19.847830865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:32.790286088 +0000 UTC m=+28.679417465" watchObservedRunningTime="2026-06-11 16:44:32.791435916 +0000 UTC m=+28.680567292" Jun 11 16:44:32.912041 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.912013 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k8t85"] Jun 11 16:44:32.912191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.912125 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:32.912250 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:32.912231 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:32.915724 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.915696 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7sddr"] Jun 11 16:44:32.915853 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.915798 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:32.915916 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:32.915899 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:32.916305 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.916265 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gncdw"] Jun 11 16:44:32.916395 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:32.916373 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:32.916481 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:32.916456 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:33.759481 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:33.759445 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="2aa9e49e2762d900b6dea6b182cdd1978a6310c481a600166e20a978d5ee0d80" exitCode=0 Jun 11 16:44:33.759937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:33.759532 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"2aa9e49e2762d900b6dea6b182cdd1978a6310c481a600166e20a978d5ee0d80"} Jun 11 16:44:34.512813 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:34.512779 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:34.513021 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:34.512828 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:34.513021 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:34.512908 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:34.513021 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:34.512955 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:34.513021 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:34.512991 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:34.513227 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:34.513064 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:35.766999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:35.766969 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="aa6a361e80687a5f80b0d1fef7f6de5039e5386b78ac0edb9302bfb4965cc8c5" exitCode=0 Jun 11 16:44:35.767354 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:35.767012 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"aa6a361e80687a5f80b0d1fef7f6de5039e5386b78ac0edb9302bfb4965cc8c5"} Jun 11 16:44:36.511512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:36.511476 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:36.511512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:36.511504 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:36.512031 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:36.511623 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:36.512031 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:36.511669 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:36.512031 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:36.511752 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:36.512031 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:36.511849 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:37.311130 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:37.310857 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:37.311130 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:37.311111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:37.311130 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.311005 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:37.311612 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.311191 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:45:09.311171004 +0000 UTC m=+65.200302368 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:44:37.311612 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.311201 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:37.311612 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.311233 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret podName:1219414a-d398-4f74-a6f2-b59466fbf0da nodeName:}" failed. No retries permitted until 2026-06-11 16:45:09.311223523 +0000 UTC m=+65.200354877 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret") pod "global-pull-secret-syncer-7sddr" (UID: "1219414a-d398-4f74-a6f2-b59466fbf0da") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:44:37.411596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:37.411544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:37.411780 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.411742 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:44:37.411780 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.411767 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:44:37.411780 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.411781 2557 projected.go:196] Error preparing data for projected volume kube-api-access-9hl7w for pod openshift-network-diagnostics/network-check-target-gncdw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:37.411933 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:37.411848 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w podName:afeaeabd-9c2e-4a40-94fc-be4c35d0f301 nodeName:}" failed. No retries permitted until 2026-06-11 16:45:09.411828588 +0000 UTC m=+65.300959963 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hl7w" (UniqueName: "kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w") pod "network-check-target-gncdw" (UID: "afeaeabd-9c2e-4a40-94fc-be4c35d0f301") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:44:38.512116 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:38.512078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:38.512565 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:38.512079 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:38.512565 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:38.512216 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:38.512565 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:38.512213 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gncdw" podUID="afeaeabd-9c2e-4a40-94fc-be4c35d0f301" Jun 11 16:44:38.512565 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:38.512313 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:44:38.512565 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:38.512375 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7sddr" podUID="1219414a-d398-4f74-a6f2-b59466fbf0da" Jun 11 16:44:38.956969 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:38.956902 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeReady" Jun 11 16:44:38.957132 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:38.957070 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 16:44:39.004533 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.004500 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:44:39.007046 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.007019 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.009431 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.009387 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w"] Jun 11 16:44:39.010274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.010246 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 16:44:39.010274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.010262 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 16:44:39.010694 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.010667 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fsrsr\"" Jun 11 16:44:39.011313 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.011268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.011427 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.011371 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 16:44:39.014238 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.014211 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 11 16:44:39.014400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.014290 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 11 16:44:39.014509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.014438 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p6n5b\"" Jun 11 16:44:39.017138 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.017116 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 16:44:39.034779 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.034451 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:44:39.035622 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.035600 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w"] Jun 11 16:44:39.044590 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.044554 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gwwn2"] Jun 11 16:44:39.047968 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.047943 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.048745 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.048726 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2dcgz"] Jun 11 16:44:39.050788 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.050768 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.054727 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.054705 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 16:44:39.054818 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.054775 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 16:44:39.066708 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.066687 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-428pv\"" Jun 11 16:44:39.067658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.067637 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d9x4d\"" Jun 11 16:44:39.072492 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.072467 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 16:44:39.073690 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.073672 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 16:44:39.073776 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.073715 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 16:44:39.093596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.093548 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwwn2"] Jun 11 16:44:39.107235 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.107165 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2dcgz"] Jun 11 16:44:39.124172 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124124 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124172 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124167 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124409 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124230 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124409 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124409 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124409 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124343 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124409 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124371 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.124644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124416 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.124644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124463 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.124644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.124520 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvk2t\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.225673 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.225673 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225703 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-cert\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225727 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssrjk\" (UniqueName: \"kubernetes.io/projected/70b96e71-be86-45c3-9d86-4f7ec5c52525-kube-api-access-ssrjk\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.225819 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.225841 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225840 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.225905 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:44:39.725885552 +0000 UTC m=+35.615016907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:39.225937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225970 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjd4\" (UniqueName: \"kubernetes.io/projected/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-kube-api-access-cpjd4\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.225993 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226034 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226061 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226091 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvk2t\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70b96e71-be86-45c3-9d86-4f7ec5c52525-tmp-dir\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.226330 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70b96e71-be86-45c3-9d86-4f7ec5c52525-config-volume\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.226769 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226512 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226769 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226512 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.226769 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.226670 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:39.226769 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.226724 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:44:39.726706534 +0000 UTC m=+35.615837890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:39.227009 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.226983 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.227490 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.227465 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.231224 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.231196 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.231350 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.231257 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.243653 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.243628 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvk2t\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.243801 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.243780 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.326899 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.326858 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70b96e71-be86-45c3-9d86-4f7ec5c52525-config-volume\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.327080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.326932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-cert\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.327080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.326958 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssrjk\" (UniqueName: \"kubernetes.io/projected/70b96e71-be86-45c3-9d86-4f7ec5c52525-kube-api-access-ssrjk\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.327080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.327017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjd4\" (UniqueName: \"kubernetes.io/projected/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-kube-api-access-cpjd4\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.327080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.327062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.327293 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.327129 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70b96e71-be86-45c3-9d86-4f7ec5c52525-tmp-dir\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.327347 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.327297 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:39.327600 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.327411 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70b96e71-be86-45c3-9d86-4f7ec5c52525-tmp-dir\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.327600 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.327445 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:39.827415895 +0000 UTC m=+35.716547250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:44:39.327600 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.327542 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70b96e71-be86-45c3-9d86-4f7ec5c52525-config-volume\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.329692 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.329671 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-cert\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.342142 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.342117 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjd4\" (UniqueName: \"kubernetes.io/projected/58ac9bf0-ee3c-4e93-a17d-b78f099d991f-kube-api-access-cpjd4\") pod \"ingress-canary-gwwn2\" (UID: \"58ac9bf0-ee3c-4e93-a17d-b78f099d991f\") " pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.343069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.343045 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssrjk\" (UniqueName: \"kubernetes.io/projected/70b96e71-be86-45c3-9d86-4f7ec5c52525-kube-api-access-ssrjk\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.360490 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.360458 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gwwn2" Jun 11 16:44:39.730216 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.730180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.730251 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.730349 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.730366 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.730415 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:44:40.730401357 +0000 UTC m=+36.619532725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.730347 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:39.730835 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.730508 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:44:40.73049501 +0000 UTC m=+36.619626378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:39.830775 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:39.830736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:39.830961 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.830901 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:39.831024 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:39.830973 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:40.830955436 +0000 UTC m=+36.720086791 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:44:40.511663 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.511572 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:44:40.511663 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.511623 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:44:40.512083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.512043 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:44:40.514364 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.514139 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:44:40.514442 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.514410 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:44:40.516161 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.516052 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mwddw\"" Jun 11 16:44:40.516280 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.516263 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:44:40.516354 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.516304 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bmxfr\"" Jun 11 16:44:40.516519 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.516501 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:44:40.567120 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.566941 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gwwn2"] Jun 11 16:44:40.575663 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:44:40.575621 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ac9bf0_ee3c_4e93_a17d_b78f099d991f.slice/crio-1d3a558bcc1954dbb4e325ec8b1087688c18fc254880d3256b73f45727f14521 WatchSource:0}: Error finding container 1d3a558bcc1954dbb4e325ec8b1087688c18fc254880d3256b73f45727f14521: Status 404 returned error can't find the container with id 1d3a558bcc1954dbb4e325ec8b1087688c18fc254880d3256b73f45727f14521 Jun 11 16:44:40.738268 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.738236 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.738309 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.738369 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.738393 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.738408 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.738459 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:44:42.738440489 +0000 UTC m=+38.627571845 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:40.738802 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.738473 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:44:42.738467166 +0000 UTC m=+38.627598521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:40.779972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.779935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwwn2" event={"ID":"58ac9bf0-ee3c-4e93-a17d-b78f099d991f","Type":"ContainerStarted","Data":"1d3a558bcc1954dbb4e325ec8b1087688c18fc254880d3256b73f45727f14521"} Jun 11 16:44:40.782473 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.782445 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="bd1106aa0675cd25a88b078b3eed5080edbaac7fe543ec40a65acbd540ee6563" exitCode=0 Jun 11 16:44:40.782618 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.782488 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"bd1106aa0675cd25a88b078b3eed5080edbaac7fe543ec40a65acbd540ee6563"} Jun 11 16:44:40.838754 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:40.838722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:40.838902 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.838833 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:40.838902 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:40.838883 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:42.838870526 +0000 UTC m=+38.728001881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:44:41.787951 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:41.787920 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b585434-323b-4b8a-9a66-edb9e17f41e1" containerID="b94421831a345cb7d269c608d29b538ea17c2d4f8d4138522cd529b5ad6c16d1" exitCode=0 Jun 11 16:44:41.788299 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:41.787982 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerDied","Data":"b94421831a345cb7d269c608d29b538ea17c2d4f8d4138522cd529b5ad6c16d1"} Jun 11 16:44:42.753718 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:42.753680 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:42.753898 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:42.753752 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:42.753898 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.753864 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:42.753898 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.753865 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:42.753898 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.753890 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:42.754097 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.753928 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:44:46.753909333 +0000 UTC m=+42.643040692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:42.754097 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.753947 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:44:46.753937446 +0000 UTC m=+42.643068801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:42.794736 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:42.794705 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" event={"ID":"6b585434-323b-4b8a-9a66-edb9e17f41e1","Type":"ContainerStarted","Data":"387bb88fe1184f55a5c73f608bedc8763621480451a4a055519ef4c76d6ee532"} Jun 11 16:44:42.819609 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:42.819525 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fxs4v" podStartSLOduration=7.939359521 podStartE2EDuration="38.81950365s" totalImagesPullingTime="30.880144129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:42.818078081 +0000 UTC m=+38.707209492" watchObservedRunningTime="2026-06-11 16:44:42.81950365 +0000 UTC m=+38.708635028" Jun 11 16:44:42.854926 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:42.854896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:42.855104 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.855077 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:42.855172 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:42.855149 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:46.855128602 +0000 UTC m=+42.744259960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:44:43.798508 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:43.798413 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gwwn2" event={"ID":"58ac9bf0-ee3c-4e93-a17d-b78f099d991f","Type":"ContainerStarted","Data":"98b88a83aa91f432690e2dbc978c7cc9d44ac2c22a1782dd9cacf922831dc4c8"} Jun 11 16:44:43.814561 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:43.814509 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gwwn2" podStartSLOduration=1.91711492 podStartE2EDuration="4.814493106s" totalImagesPullingTime="2.897378186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:44:43.814265512 +0000 UTC m=+39.703396896" watchObservedRunningTime="2026-06-11 16:44:43.814493106 +0000 UTC m=+39.703624483" Jun 11 16:44:46.787182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:46.787138 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:46.787207 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.787297 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.787310 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.787322 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.787365 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:44:54.787348473 +0000 UTC m=+50.676479827 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:46.787554 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.787380 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:44:54.78737281 +0000 UTC m=+50.676504164 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:46.888303 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:46.888273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:46.888470 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.888425 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:46.888507 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:46.888489 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:44:54.888473333 +0000 UTC m=+50.777604688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:44:54.844109 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:54.844075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:54.844146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.844239 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.844263 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.844235 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.844322 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:45:10.844307533 +0000 UTC m=+66.733438894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:44:54.844657 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.844337 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:45:10.844331707 +0000 UTC m=+66.733463064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:44:54.944760 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:44:54.944724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:44:54.944853 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.944837 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:44:54.944912 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:44:54.944902 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:45:10.94488553 +0000 UTC m=+66.834016906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:45:04.774848 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:04.774819 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s8cmj" Jun 11 16:45:09.349531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.349471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:45:09.350083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.349594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:45:09.352180 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.352156 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:45:09.352307 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.352164 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:45:09.360127 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:09.360109 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:45:09.360195 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:09.360185 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:46:13.360171735 +0000 UTC m=+129.249303090 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : secret "metrics-daemon-secret" not found Jun 11 16:45:09.363438 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.363421 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1219414a-d398-4f74-a6f2-b59466fbf0da-original-pull-secret\") pod \"global-pull-secret-syncer-7sddr\" (UID: \"1219414a-d398-4f74-a6f2-b59466fbf0da\") " pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:45:09.450932 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.450897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:45:09.453949 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.453927 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:45:09.464525 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.464502 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:45:09.474988 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.474961 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hl7w\" (UniqueName: \"kubernetes.io/projected/afeaeabd-9c2e-4a40-94fc-be4c35d0f301-kube-api-access-9hl7w\") pod \"network-check-target-gncdw\" (UID: \"afeaeabd-9c2e-4a40-94fc-be4c35d0f301\") " pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:45:09.630599 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.630485 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mwddw\"" Jun 11 16:45:09.636794 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.636768 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7sddr" Jun 11 16:45:09.638902 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.638884 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:45:09.796557 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.796528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gncdw"] Jun 11 16:45:09.802463 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:45:09.800108 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafeaeabd_9c2e_4a40_94fc_be4c35d0f301.slice/crio-8cea210de5a93f50886670ce3f0befb971e7660403f374963bd9be68e73ca913 WatchSource:0}: Error finding container 8cea210de5a93f50886670ce3f0befb971e7660403f374963bd9be68e73ca913: Status 404 returned error can't find the container with id 8cea210de5a93f50886670ce3f0befb971e7660403f374963bd9be68e73ca913 Jun 11 16:45:09.813148 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.813126 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7sddr"] Jun 11 16:45:09.815724 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:45:09.815699 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1219414a_d398_4f74_a6f2_b59466fbf0da.slice/crio-3c4e923dc9ed8dd2b843294efa2d6075ce502cf0190de0830c0fcccc1c3c9a35 WatchSource:0}: Error finding container 3c4e923dc9ed8dd2b843294efa2d6075ce502cf0190de0830c0fcccc1c3c9a35: Status 404 returned error can't find the container with id 3c4e923dc9ed8dd2b843294efa2d6075ce502cf0190de0830c0fcccc1c3c9a35 Jun 11 16:45:09.858291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.858262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gncdw" event={"ID":"afeaeabd-9c2e-4a40-94fc-be4c35d0f301","Type":"ContainerStarted","Data":"8cea210de5a93f50886670ce3f0befb971e7660403f374963bd9be68e73ca913"} Jun 11 16:45:09.859398 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:09.859373 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7sddr" event={"ID":"1219414a-d398-4f74-a6f2-b59466fbf0da","Type":"ContainerStarted","Data":"3c4e923dc9ed8dd2b843294efa2d6075ce502cf0190de0830c0fcccc1c3c9a35"} Jun 11 16:45:10.862572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:10.862531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:10.862642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.862745 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.862779 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.862801 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:45:42.862783407 +0000 UTC m=+98.751914764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.862801 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:45:10.863057 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.862873 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:45:42.862852178 +0000 UTC m=+98.751983547 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:45:10.963509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:10.963471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:45:10.963731 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.963684 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:45:10.963800 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:10.963772 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:45:42.963742491 +0000 UTC m=+98.852873850 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:45:14.875192 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:14.875151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gncdw" event={"ID":"afeaeabd-9c2e-4a40-94fc-be4c35d0f301","Type":"ContainerStarted","Data":"1bf1914ec5aef3f1a0ad4f621233b43b97eb8081d1af4c70fae7b6870a9e3f6e"} Jun 11 16:45:14.875670 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:14.875242 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:45:14.876655 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:14.876630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7sddr" event={"ID":"1219414a-d398-4f74-a6f2-b59466fbf0da","Type":"ContainerStarted","Data":"5b854d2a9c8b8ec8d8b739e390c92cdd3a76e7855d944f8222a2b71b68aca190"} Jun 11 16:45:14.891276 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:14.891205 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gncdw" podStartSLOduration=66.377006325 podStartE2EDuration="1m10.891188674s" totalImagesPullingTime="4.514182349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:45:14.891152308 +0000 UTC m=+70.780283685" watchObservedRunningTime="2026-06-11 16:45:14.891188674 +0000 UTC m=+70.780320049" Jun 11 16:45:14.910892 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:14.910842 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7sddr" podStartSLOduration=66.398901467 podStartE2EDuration="1m10.910826018s" totalImagesPullingTime="4.511924551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:45:14.909928483 +0000 UTC m=+70.799059893" watchObservedRunningTime="2026-06-11 16:45:14.910826018 +0000 UTC m=+70.799957423" Jun 11 16:45:42.890962 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:42.890920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.891056 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.891074 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:42.891073 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.891133 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:46:46.891119277 +0000 UTC m=+162.780250631 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.891172 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:45:42.891373 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.891221 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:46:46.891210039 +0000 UTC m=+162.780341394 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:45:42.992362 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:42.992318 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:45:42.992528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.992465 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:45:42.992528 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:45:42.992528 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:46:46.992512745 +0000 UTC m=+162.881644100 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:45:45.880848 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:45:45.880818 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gncdw" Jun 11 16:46:13.410334 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:13.410280 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:46:13.410879 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:13.410434 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:46:13.410879 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:13.410498 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:15.410482714 +0000 UTC m=+251.299614072 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : secret "metrics-daemon-secret" not found Jun 11 16:46:42.020417 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:42.020357 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" Jun 11 16:46:42.027546 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:42.027508 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" podUID="55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e" Jun 11 16:46:42.064825 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:42.064781 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2dcgz" podUID="70b96e71-be86-45c3-9d86-4f7ec5c52525" Jun 11 16:46:42.084566 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:42.084541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:46:42.084708 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:42.084541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:46:42.084748 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:42.084541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:46:43.544151 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:43.544109 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:46:46.937118 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:46.937085 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:46.937143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:46.937255 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:46.937269 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c595d8856-r7dsx: secret "image-registry-tls" not found Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:46.937317 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls podName:9de10675-cbff-4637-93ef-a69149df0c7b nodeName:}" failed. No retries permitted until 2026-06-11 16:48:48.937303056 +0000 UTC m=+284.826434411 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls") pod "image-registry-c595d8856-r7dsx" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b") : secret "image-registry-tls" not found Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:46.937252 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:46:46.937535 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:46.937421 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert podName:55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e nodeName:}" failed. No retries permitted until 2026-06-11 16:48:48.937404498 +0000 UTC m=+284.826535853 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-6pw9w" (UID: "55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e") : secret "networking-console-plugin-cert" not found Jun 11 16:46:47.038176 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:47.038142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:46:47.038333 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:47.038259 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:46:47.038333 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:46:47.038309 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls podName:70b96e71-be86-45c3-9d86-4f7ec5c52525 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:49.038297083 +0000 UTC m=+284.927428438 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls") pod "dns-default-2dcgz" (UID: "70b96e71-be86-45c3-9d86-4f7ec5c52525") : secret "dns-default-metrics-tls" not found Jun 11 16:46:56.511888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:46:56.511787 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:48:15.464252 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:15.464201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:48:15.464770 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:15.464369 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:48:15.464770 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:15.464443 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs podName:b05df1bd-4184-445d-be31-e2364f0f5079 nodeName:}" failed. No retries permitted until 2026-06-11 16:50:17.464427025 +0000 UTC m=+373.353558385 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs") pod "network-metrics-daemon-k8t85" (UID: "b05df1bd-4184-445d-be31-e2364f0f5079") : secret "metrics-daemon-secret" not found Jun 11 16:48:30.672331 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.672251 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb"] Jun 11 16:48:30.674232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.674215 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" Jun 11 16:48:30.677511 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.677489 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-krkxc\"" Jun 11 16:48:30.683709 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.683693 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2"] Jun 11 16:48:30.685455 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.685440 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99"] Jun 11 16:48:30.685544 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.685524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.687747 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.687730 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr"] Jun 11 16:48:30.687867 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.687852 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" Jun 11 16:48:30.689764 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.689743 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6ddbb9c9cc-g52gs"] Jun 11 16:48:30.689932 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.689916 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.690052 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.690037 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 11 16:48:30.691542 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.691524 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.693629 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.693612 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:48:30.693798 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.693780 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 11 16:48:30.693895 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.693879 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-228s4\"" Jun 11 16:48:30.693945 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.693919 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-w9l5g\"" Jun 11 16:48:30.693991 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.693969 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:48:30.694478 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.694463 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 11 16:48:30.695614 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695599 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 11 16:48:30.695700 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695655 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 11 16:48:30.695700 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695604 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 11 16:48:30.695807 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695683 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:48:30.695807 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695747 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 11 16:48:30.695882 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695866 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 11 16:48:30.695999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.695981 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-p59vl\"" Jun 11 16:48:30.696087 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.696074 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 11 16:48:30.697367 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.697351 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 16:48:30.697500 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.697388 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 16:48:30.698450 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.698418 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2lg7c\"" Jun 11 16:48:30.698938 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.698917 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 11 16:48:30.714325 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.714305 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2"] Jun 11 16:48:30.718702 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.718681 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99"] Jun 11 16:48:30.725403 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.725370 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr"] Jun 11 16:48:30.726194 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.726175 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ddbb9c9cc-g52gs"] Jun 11 16:48:30.740930 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.740903 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb"] Jun 11 16:48:30.774208 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774175 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctj55\" (UniqueName: \"kubernetes.io/projected/7b91dede-ffde-4c85-a5a7-382cc937f42a-kube-api-access-ctj55\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.774381 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774241 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-stats-auth\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.774381 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzrg\" (UniqueName: \"kubernetes.io/projected/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-kube-api-access-dmzrg\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.774381 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774367 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774402 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4kkr\" (UniqueName: \"kubernetes.io/projected/20566839-8048-4bc9-9782-ab17969e59b1-kube-api-access-d4kkr\") pod \"network-check-source-dbd9cf844-gcqcb\" (UID: \"20566839-8048-4bc9-9782-ab17969e59b1\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774433 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dccdj\" (UniqueName: \"kubernetes.io/projected/e828a0bd-57dc-4224-a2c4-2ac58e5c09d3-kube-api-access-dccdj\") pod \"volume-data-source-validator-78dd696465-6gm99\" (UID: \"e828a0bd-57dc-4224-a2c4-2ac58e5c09d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-default-certificate\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774521 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwml\" (UniqueName: \"kubernetes.io/projected/3e0e805a-91d5-4248-9610-d31e52f3c821-kube-api-access-8mwml\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.774558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774547 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.774799 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774572 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.774799 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.774625 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.875147 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwml\" (UniqueName: \"kubernetes.io/projected/3e0e805a-91d5-4248-9610-d31e52f3c821-kube-api-access-8mwml\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.875147 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875148 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875169 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875185 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875215 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctj55\" (UniqueName: \"kubernetes.io/projected/7b91dede-ffde-4c85-a5a7-382cc937f42a-kube-api-access-ctj55\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875259 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-stats-auth\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:30.875284 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875287 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzrg\" (UniqueName: \"kubernetes.io/projected/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-kube-api-access-dmzrg\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:30.875327 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:31.37530593 +0000 UTC m=+267.264437288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : configmap references non-existent config key: service-ca.crt Jun 11 16:48:30.875393 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:30.875383 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls podName:3e0e805a-91d5-4248-9610-d31e52f3c821 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:31.37536426 +0000 UTC m=+267.264495623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gzwx2" (UID: "3e0e805a-91d5-4248-9610-d31e52f3c821") : secret "samples-operator-tls" not found Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875439 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875477 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4kkr\" (UniqueName: \"kubernetes.io/projected/20566839-8048-4bc9-9782-ab17969e59b1-kube-api-access-d4kkr\") pod \"network-check-source-dbd9cf844-gcqcb\" (UID: \"20566839-8048-4bc9-9782-ab17969e59b1\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875507 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dccdj\" (UniqueName: \"kubernetes.io/projected/e828a0bd-57dc-4224-a2c4-2ac58e5c09d3-kube-api-access-dccdj\") pod \"volume-data-source-validator-78dd696465-6gm99\" (UID: \"e828a0bd-57dc-4224-a2c4-2ac58e5c09d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-default-certificate\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:30.875685 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:30.875730 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:31.375716335 +0000 UTC m=+267.264847696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : secret "router-metrics-certs-default" not found Jun 11 16:48:30.875824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.875819 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.877802 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.877767 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.877950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.877929 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-default-certificate\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.878053 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.877939 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-stats-auth\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.897400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.897377 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctj55\" (UniqueName: \"kubernetes.io/projected/7b91dede-ffde-4c85-a5a7-382cc937f42a-kube-api-access-ctj55\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:30.908287 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.908260 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzrg\" (UniqueName: \"kubernetes.io/projected/6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11-kube-api-access-dmzrg\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-r8gzr\" (UID: \"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:30.908820 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.908637 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4kkr\" (UniqueName: \"kubernetes.io/projected/20566839-8048-4bc9-9782-ab17969e59b1-kube-api-access-d4kkr\") pod \"network-check-source-dbd9cf844-gcqcb\" (UID: \"20566839-8048-4bc9-9782-ab17969e59b1\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" Jun 11 16:48:30.908820 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.908772 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dccdj\" (UniqueName: \"kubernetes.io/projected/e828a0bd-57dc-4224-a2c4-2ac58e5c09d3-kube-api-access-dccdj\") pod \"volume-data-source-validator-78dd696465-6gm99\" (UID: \"e828a0bd-57dc-4224-a2c4-2ac58e5c09d3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" Jun 11 16:48:30.908933 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.908886 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwml\" (UniqueName: \"kubernetes.io/projected/3e0e805a-91d5-4248-9610-d31e52f3c821-kube-api-access-8mwml\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:30.982807 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:30.982774 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" Jun 11 16:48:31.001757 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.001727 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" Jun 11 16:48:31.008943 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.008914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" Jun 11 16:48:31.133730 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.133699 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb"] Jun 11 16:48:31.138494 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:31.138462 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20566839_8048_4bc9_9782_ab17969e59b1.slice/crio-9242d4be5a7a9c3a1e0bab5cdbb4d07bb447892fc40f868dd5b341e49997e6e1 WatchSource:0}: Error finding container 9242d4be5a7a9c3a1e0bab5cdbb4d07bb447892fc40f868dd5b341e49997e6e1: Status 404 returned error can't find the container with id 9242d4be5a7a9c3a1e0bab5cdbb4d07bb447892fc40f868dd5b341e49997e6e1 Jun 11 16:48:31.158005 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.157977 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99"] Jun 11 16:48:31.174921 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:31.174881 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode828a0bd_57dc_4224_a2c4_2ac58e5c09d3.slice/crio-a383e6218c7e6ecb82b7381e404b977cc231cd7a69166d8ebe71ec9e117be224 WatchSource:0}: Error finding container a383e6218c7e6ecb82b7381e404b977cc231cd7a69166d8ebe71ec9e117be224: Status 404 returned error can't find the container with id a383e6218c7e6ecb82b7381e404b977cc231cd7a69166d8ebe71ec9e117be224 Jun 11 16:48:31.175805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.175784 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr"] Jun 11 16:48:31.178638 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:31.178616 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b70d9e6_fc73_4f87_bb35_b79b2ab0ff11.slice/crio-d392e12b568612268c4579d016876a6194489147074cb3d1898307f533119c03 WatchSource:0}: Error finding container d392e12b568612268c4579d016876a6194489147074cb3d1898307f533119c03: Status 404 returned error can't find the container with id d392e12b568612268c4579d016876a6194489147074cb3d1898307f533119c03 Jun 11 16:48:31.338892 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.338793 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" event={"ID":"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11","Type":"ContainerStarted","Data":"d392e12b568612268c4579d016876a6194489147074cb3d1898307f533119c03"} Jun 11 16:48:31.339877 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.339850 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" event={"ID":"e828a0bd-57dc-4224-a2c4-2ac58e5c09d3","Type":"ContainerStarted","Data":"a383e6218c7e6ecb82b7381e404b977cc231cd7a69166d8ebe71ec9e117be224"} Jun 11 16:48:31.341142 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.341110 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" event={"ID":"20566839-8048-4bc9-9782-ab17969e59b1","Type":"ContainerStarted","Data":"40d23f613efc92ff34bd7f0a801dde96263212ee5479562b36b28316d3b81880"} Jun 11 16:48:31.341218 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.341162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" event={"ID":"20566839-8048-4bc9-9782-ab17969e59b1","Type":"ContainerStarted","Data":"9242d4be5a7a9c3a1e0bab5cdbb4d07bb447892fc40f868dd5b341e49997e6e1"} Jun 11 16:48:31.380785 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.380756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:31.380921 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.380793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:31.380921 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:31.380897 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:32.380879205 +0000 UTC m=+268.270010561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : configmap references non-existent config key: service-ca.crt Jun 11 16:48:31.380921 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:31.380911 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:48:31.381075 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:31.380927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:31.381075 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:31.380969 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:32.380950714 +0000 UTC m=+268.270082071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : secret "router-metrics-certs-default" not found Jun 11 16:48:31.381075 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:31.381015 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:48:31.381075 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:31.381053 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls podName:3e0e805a-91d5-4248-9610-d31e52f3c821 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:32.38104357 +0000 UTC m=+268.270174926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gzwx2" (UID: "3e0e805a-91d5-4248-9610-d31e52f3c821") : secret "samples-operator-tls" not found Jun 11 16:48:32.351530 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.351487 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" event={"ID":"e828a0bd-57dc-4224-a2c4-2ac58e5c09d3","Type":"ContainerStarted","Data":"79a1bad0af67286148cba4e9555d2fb70d4696cc95b396597efcec5f53d453c3"} Jun 11 16:48:32.370647 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.370572 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-6gm99" podStartSLOduration=1.3601870649999999 podStartE2EDuration="2.370556091s" totalImagesPullingTime="1.010369026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:32.369481507 +0000 UTC m=+268.258612888" watchObservedRunningTime="2026-06-11 16:48:32.370556091 +0000 UTC m=+268.259687471" Jun 11 16:48:32.370847 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.370821 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-gcqcb" podStartSLOduration=2.370806241 podStartE2EDuration="2.370806241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:31.364417366 +0000 UTC m=+267.253548743" watchObservedRunningTime="2026-06-11 16:48:32.370806241 +0000 UTC m=+268.259937618" Jun 11 16:48:32.388620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.388562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:32.388768 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:32.388670 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:48:32.388768 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:32.388735 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:34.388720645 +0000 UTC m=+270.277852001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : secret "router-metrics-certs-default" not found Jun 11 16:48:32.388768 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.388727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:32.388768 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:32.388763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:32.388967 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:32.388830 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:34.388813576 +0000 UTC m=+270.277944934 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : configmap references non-existent config key: service-ca.crt Jun 11 16:48:32.388967 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:32.388860 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:48:32.388967 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:32.388913 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls podName:3e0e805a-91d5-4248-9610-d31e52f3c821 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:34.388900499 +0000 UTC m=+270.278031855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gzwx2" (UID: "3e0e805a-91d5-4248-9610-d31e52f3c821") : secret "samples-operator-tls" not found Jun 11 16:48:33.358050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:33.357952 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" event={"ID":"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11","Type":"ContainerStarted","Data":"db42c4be8ba0761a3aea975360e365c540ce88535b5664807e2eab876aac06d8"} Jun 11 16:48:33.375034 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:33.374974 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" podStartSLOduration=1.5629624039999999 podStartE2EDuration="3.37495406s" totalImagesPullingTime="1.811991656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:33.373884496 +0000 UTC m=+269.263015866" watchObservedRunningTime="2026-06-11 16:48:33.37495406 +0000 UTC m=+269.264085439" Jun 11 16:48:34.407053 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.407019 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.407162 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:34.407238 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:34.407283 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:38.407268727 +0000 UTC m=+274.296400083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : secret "router-metrics-certs-default" not found Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.407233 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:34.407308 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:38.407287239 +0000 UTC m=+274.296418603 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : configmap references non-existent config key: service-ca.crt Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:34.407324 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:48:34.407512 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:34.407382 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls podName:3e0e805a-91d5-4248-9610-d31e52f3c821 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:38.40736842 +0000 UTC m=+274.296499775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gzwx2" (UID: "3e0e805a-91d5-4248-9610-d31e52f3c821") : secret "samples-operator-tls" not found Jun 11 16:48:34.488724 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.488686 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672"] Jun 11 16:48:34.490643 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.490628 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" Jun 11 16:48:34.493416 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.493382 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 11 16:48:34.493706 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.493682 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 11 16:48:34.494125 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.494107 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5b26z\"" Jun 11 16:48:34.507115 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.507089 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672"] Jun 11 16:48:34.609366 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.609325 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmrgx\" (UniqueName: \"kubernetes.io/projected/eb7b202c-5547-4548-91e8-c3c9679f12cc-kube-api-access-xmrgx\") pod \"migrator-955ffdf5f-6r672\" (UID: \"eb7b202c-5547-4548-91e8-c3c9679f12cc\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" Jun 11 16:48:34.710566 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.710483 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmrgx\" (UniqueName: \"kubernetes.io/projected/eb7b202c-5547-4548-91e8-c3c9679f12cc-kube-api-access-xmrgx\") pod \"migrator-955ffdf5f-6r672\" (UID: \"eb7b202c-5547-4548-91e8-c3c9679f12cc\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" Jun 11 16:48:34.718999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.718967 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmrgx\" (UniqueName: \"kubernetes.io/projected/eb7b202c-5547-4548-91e8-c3c9679f12cc-kube-api-access-xmrgx\") pod \"migrator-955ffdf5f-6r672\" (UID: \"eb7b202c-5547-4548-91e8-c3c9679f12cc\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" Jun 11 16:48:34.799549 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.799505 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" Jun 11 16:48:34.921698 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:34.921677 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672"] Jun 11 16:48:34.924000 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:34.923960 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7b202c_5547_4548_91e8_c3c9679f12cc.slice/crio-8604aad1da769d101f2db395a0d7a107c0c04061ef5b4407c2734510d7c9de63 WatchSource:0}: Error finding container 8604aad1da769d101f2db395a0d7a107c0c04061ef5b4407c2734510d7c9de63: Status 404 returned error can't find the container with id 8604aad1da769d101f2db395a0d7a107c0c04061ef5b4407c2734510d7c9de63 Jun 11 16:48:35.369125 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:35.369083 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" event={"ID":"eb7b202c-5547-4548-91e8-c3c9679f12cc","Type":"ContainerStarted","Data":"8604aad1da769d101f2db395a0d7a107c0c04061ef5b4407c2734510d7c9de63"} Jun 11 16:48:36.374074 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:36.373987 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" event={"ID":"eb7b202c-5547-4548-91e8-c3c9679f12cc","Type":"ContainerStarted","Data":"8cc26570f36f987fa0342e5ef764d557ad53abea2f5f6b2211d02ce7ab4a7606"} Jun 11 16:48:36.374074 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:36.374040 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" event={"ID":"eb7b202c-5547-4548-91e8-c3c9679f12cc","Type":"ContainerStarted","Data":"6355894b29d810b42eaffb5bb5143d31775c05877c14b90e5a60bcb2efc5e873"} Jun 11 16:48:36.391073 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:36.391029 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-6r672" podStartSLOduration=1.296500642 podStartE2EDuration="2.391013169s" totalImagesPullingTime="1.094512527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:36.389799399 +0000 UTC m=+272.278930777" watchObservedRunningTime="2026-06-11 16:48:36.391013169 +0000 UTC m=+272.280144595" Jun 11 16:48:38.442999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:38.442945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:38.442999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:38.443003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:38.443031 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:38.443101 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:38.443141 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:46.443123491 +0000 UTC m=+282.332254854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : configmap references non-existent config key: service-ca.crt Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:38.443161 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs podName:7b91dede-ffde-4c85-a5a7-382cc937f42a nodeName:}" failed. No retries permitted until 2026-06-11 16:48:46.443155571 +0000 UTC m=+282.332286926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs") pod "router-default-6ddbb9c9cc-g52gs" (UID: "7b91dede-ffde-4c85-a5a7-382cc937f42a") : secret "router-metrics-certs-default" not found Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:38.443193 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:48:38.443487 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:38.443244 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls podName:3e0e805a-91d5-4248-9610-d31e52f3c821 nodeName:}" failed. No retries permitted until 2026-06-11 16:48:46.443230879 +0000 UTC m=+282.332362233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gzwx2" (UID: "3e0e805a-91d5-4248-9610-d31e52f3c821") : secret "samples-operator-tls" not found Jun 11 16:48:38.453126 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:38.453106 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k2nzc_4091c34c-6597-4aef-9417-1aa198593e6f/dns-node-resolver/0.log" Jun 11 16:48:39.254505 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:39.254478 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bjlhc_d6678642-6849-4cda-9772-fcf28a9eb640/node-ca/0.log" Jun 11 16:48:40.251669 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:40.251641 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwwn2_58ac9bf0-ee3c-4e93-a17d-b78f099d991f/serve-healthcheck-canary/0.log" Jun 11 16:48:40.651461 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:40.651379 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-6r672_eb7b202c-5547-4548-91e8-c3c9679f12cc/migrator/0.log" Jun 11 16:48:40.850509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:40.850479 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-6r672_eb7b202c-5547-4548-91e8-c3c9679f12cc/graceful-termination/0.log" Jun 11 16:48:41.052284 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:41.052253 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-r8gzr_6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11/kube-storage-version-migrator-operator/0.log" Jun 11 16:48:45.085635 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:45.085595 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" podUID="55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e" Jun 11 16:48:45.085635 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:45.085607 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" Jun 11 16:48:45.085635 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:45.085607 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2dcgz" podUID="70b96e71-be86-45c3-9d86-4f7ec5c52525" Jun 11 16:48:45.400952 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:45.400870 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:48:45.401077 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:45.400870 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:48:45.401077 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:45.400876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:48:46.514166 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.514132 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:46.514166 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.514172 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:46.514791 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.514270 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:46.514849 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.514786 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b91dede-ffde-4c85-a5a7-382cc937f42a-service-ca-bundle\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:46.516634 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.516610 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0e805a-91d5-4248-9610-d31e52f3c821-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gzwx2\" (UID: \"3e0e805a-91d5-4248-9610-d31e52f3c821\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:46.516751 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.516735 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b91dede-ffde-4c85-a5a7-382cc937f42a-metrics-certs\") pod \"router-default-6ddbb9c9cc-g52gs\" (UID: \"7b91dede-ffde-4c85-a5a7-382cc937f42a\") " pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:46.596365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.596332 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" Jun 11 16:48:46.613177 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.613152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:46.738190 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.738156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2"] Jun 11 16:48:46.755358 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:46.755327 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ddbb9c9cc-g52gs"] Jun 11 16:48:46.758142 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:46.758116 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b91dede_ffde_4c85_a5a7_382cc937f42a.slice/crio-e6b7703d6dbef2156fe7c43bbd72a8cdf2f60b0e2e67574c0b4727c1cf5e00a5 WatchSource:0}: Error finding container e6b7703d6dbef2156fe7c43bbd72a8cdf2f60b0e2e67574c0b4727c1cf5e00a5: Status 404 returned error can't find the container with id e6b7703d6dbef2156fe7c43bbd72a8cdf2f60b0e2e67574c0b4727c1cf5e00a5 Jun 11 16:48:47.408651 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.408610 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" event={"ID":"3e0e805a-91d5-4248-9610-d31e52f3c821","Type":"ContainerStarted","Data":"5158b0fb773b7075d89b6b680382aaa6135936213c894e58c1147c44a6413744"} Jun 11 16:48:47.410249 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.410220 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" event={"ID":"7b91dede-ffde-4c85-a5a7-382cc937f42a","Type":"ContainerStarted","Data":"ae7e96f1719c43dcafba869d13261401668b88c039a217b8775b2e1ae1d0f793"} Jun 11 16:48:47.410416 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.410399 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" event={"ID":"7b91dede-ffde-4c85-a5a7-382cc937f42a","Type":"ContainerStarted","Data":"e6b7703d6dbef2156fe7c43bbd72a8cdf2f60b0e2e67574c0b4727c1cf5e00a5"} Jun 11 16:48:47.433305 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.433250 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" podStartSLOduration=17.433221997 podStartE2EDuration="17.433221997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:47.43121975 +0000 UTC m=+283.320351129" watchObservedRunningTime="2026-06-11 16:48:47.433221997 +0000 UTC m=+283.322353375" Jun 11 16:48:47.614255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.614218 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:47.618347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:47.618322 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:48.414653 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:48.414600 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:48.416170 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:48.416134 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6ddbb9c9cc-g52gs" Jun 11 16:48:49.035060 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.035021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:48:49.035537 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.035195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:48:49.037591 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.037549 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"image-registry-c595d8856-r7dsx\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:48:49.037693 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.037625 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-6pw9w\" (UID: \"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:48:49.136390 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.136347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:48:49.138645 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.138626 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70b96e71-be86-45c3-9d86-4f7ec5c52525-metrics-tls\") pod \"dns-default-2dcgz\" (UID: \"70b96e71-be86-45c3-9d86-4f7ec5c52525\") " pod="openshift-dns/dns-default-2dcgz" Jun 11 16:48:49.304505 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.304425 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fsrsr\"" Jun 11 16:48:49.304664 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.304512 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d9x4d\"" Jun 11 16:48:49.304664 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.304642 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p6n5b\"" Jun 11 16:48:49.312367 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.312350 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:48:49.312431 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.312382 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:48:49.312469 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.312435 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" Jun 11 16:48:49.419671 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.419609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" event={"ID":"3e0e805a-91d5-4248-9610-d31e52f3c821","Type":"ContainerStarted","Data":"585d7dc7da9d932bf617ab3c51a1a83551d71a18ecfebd91894c94d04ca4f990"} Jun 11 16:48:49.419671 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.419669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" event={"ID":"3e0e805a-91d5-4248-9610-d31e52f3c821","Type":"ContainerStarted","Data":"963f5701ef712b2b3056fdf70bb3705734b16411133bfbf31f0d8e4fcb5e77a0"} Jun 11 16:48:49.438324 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.438270 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gzwx2" podStartSLOduration=17.753841421 podStartE2EDuration="19.438250742s" totalImagesPullingTime="1.684409321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:49.437731605 +0000 UTC m=+285.326862986" watchObservedRunningTime="2026-06-11 16:48:49.438250742 +0000 UTC m=+285.327382119" Jun 11 16:48:49.461973 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.461939 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:48:49.465410 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:49.465382 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de10675_cbff_4637_93ef_a69149df0c7b.slice/crio-c2770ab6f5f68099655c7b2108dd8cf5d7bd0c629e51a49225b20971a2ccde1f WatchSource:0}: Error finding container c2770ab6f5f68099655c7b2108dd8cf5d7bd0c629e51a49225b20971a2ccde1f: Status 404 returned error can't find the container with id c2770ab6f5f68099655c7b2108dd8cf5d7bd0c629e51a49225b20971a2ccde1f Jun 11 16:48:49.679821 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.679625 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w"] Jun 11 16:48:49.682666 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:49.682630 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fa74ba_8ddc_4f2e_82f1_2ef5e3238a8e.slice/crio-9d127adbe4355a277c4169cfc86e0181ff8d07d0e21bb04a31144f5c060720da WatchSource:0}: Error finding container 9d127adbe4355a277c4169cfc86e0181ff8d07d0e21bb04a31144f5c060720da: Status 404 returned error can't find the container with id 9d127adbe4355a277c4169cfc86e0181ff8d07d0e21bb04a31144f5c060720da Jun 11 16:48:49.683695 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:49.683673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2dcgz"] Jun 11 16:48:49.686428 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:48:49.686386 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70b96e71_be86_45c3_9d86_4f7ec5c52525.slice/crio-2aee01150c87d0a6105419de853943a106477b42826f70f8ac45e8ed6331c755 WatchSource:0}: Error finding container 2aee01150c87d0a6105419de853943a106477b42826f70f8ac45e8ed6331c755: Status 404 returned error can't find the container with id 2aee01150c87d0a6105419de853943a106477b42826f70f8ac45e8ed6331c755 Jun 11 16:48:50.424684 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:50.424643 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" event={"ID":"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e","Type":"ContainerStarted","Data":"9d127adbe4355a277c4169cfc86e0181ff8d07d0e21bb04a31144f5c060720da"} Jun 11 16:48:50.425893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:50.425862 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dcgz" event={"ID":"70b96e71-be86-45c3-9d86-4f7ec5c52525","Type":"ContainerStarted","Data":"2aee01150c87d0a6105419de853943a106477b42826f70f8ac45e8ed6331c755"} Jun 11 16:48:50.427391 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:50.427360 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" event={"ID":"9de10675-cbff-4637-93ef-a69149df0c7b","Type":"ContainerStarted","Data":"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca"} Jun 11 16:48:50.427515 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:50.427413 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" event={"ID":"9de10675-cbff-4637-93ef-a69149df0c7b","Type":"ContainerStarted","Data":"c2770ab6f5f68099655c7b2108dd8cf5d7bd0c629e51a49225b20971a2ccde1f"} Jun 11 16:48:50.448544 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:50.448495 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" podStartSLOduration=285.448478609 podStartE2EDuration="4m45.448478609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:50.447247524 +0000 UTC m=+286.336378901" watchObservedRunningTime="2026-06-11 16:48:50.448478609 +0000 UTC m=+286.337609987" Jun 11 16:48:51.432265 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:51.432224 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:48:52.436396 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.436359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" event={"ID":"55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e","Type":"ContainerStarted","Data":"21679743af587efab5c2f02ad79319e86a24e5b70319bee41cb76c8f7b16c2c4"} Jun 11 16:48:52.438224 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.438198 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dcgz" event={"ID":"70b96e71-be86-45c3-9d86-4f7ec5c52525","Type":"ContainerStarted","Data":"1c72a3974657f16cad71dfceb2866daa88aec77b8c1c09a2d20ed91c41aa9b30"} Jun 11 16:48:52.438341 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.438245 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:48:52.438341 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.438262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2dcgz" event={"ID":"70b96e71-be86-45c3-9d86-4f7ec5c52525","Type":"ContainerStarted","Data":"d4635aa4b69fd45fd164777832056854c2b21f1cc4797fd3257ea978d7e5d1e8"} Jun 11 16:48:52.452597 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.452534 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-6pw9w" podStartSLOduration=272.643048733 podStartE2EDuration="4m34.452521212s" totalImagesPullingTime="1.809472479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:52.452210292 +0000 UTC m=+288.341341671" watchObservedRunningTime="2026-06-11 16:48:52.452521212 +0000 UTC m=+288.341652589" Jun 11 16:48:52.470129 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:48:52.470087 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2dcgz" podStartSLOduration=251.660331413 podStartE2EDuration="4m13.470071585s" totalImagesPullingTime="1.809740172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:48:52.468776155 +0000 UTC m=+288.357907532" watchObservedRunningTime="2026-06-11 16:48:52.470071585 +0000 UTC m=+288.359202961" Jun 11 16:48:59.513195 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:48:59.513154 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-k8t85" podUID="b05df1bd-4184-445d-be31-e2364f0f5079" Jun 11 16:49:01.831026 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.830976 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ksrjf"] Jun 11 16:49:01.836035 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.836013 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:01.840135 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.840113 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 16:49:01.840948 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.840929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4vcj6\"" Jun 11 16:49:01.841073 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.840975 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 16:49:01.841073 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.840982 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 16:49:01.841073 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.840929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 16:49:01.859474 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.859438 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ksrjf"] Jun 11 16:49:01.929783 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.929750 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a426988b-e088-4524-8df0-21f725d06d4e-crio-socket\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:01.929783 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.929788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a426988b-e088-4524-8df0-21f725d06d4e-data-volume\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:01.929993 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.929815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xgpm\" (UniqueName: \"kubernetes.io/projected/a426988b-e088-4524-8df0-21f725d06d4e-kube-api-access-9xgpm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:01.929993 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.929899 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a426988b-e088-4524-8df0-21f725d06d4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:01.929993 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:01.929960 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a426988b-e088-4524-8df0-21f725d06d4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.030852 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.030815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a426988b-e088-4524-8df0-21f725d06d4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.030852 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.030856 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a426988b-e088-4524-8df0-21f725d06d4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031092 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.030974 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a426988b-e088-4524-8df0-21f725d06d4e-crio-socket\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031092 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.031020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a426988b-e088-4524-8df0-21f725d06d4e-data-volume\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031092 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.031044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xgpm\" (UniqueName: \"kubernetes.io/projected/a426988b-e088-4524-8df0-21f725d06d4e-kube-api-access-9xgpm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031092 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.031082 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a426988b-e088-4524-8df0-21f725d06d4e-crio-socket\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031354 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.031334 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a426988b-e088-4524-8df0-21f725d06d4e-data-volume\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.031405 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.031372 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a426988b-e088-4524-8df0-21f725d06d4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.033352 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.033332 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a426988b-e088-4524-8df0-21f725d06d4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.049060 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.049041 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xgpm\" (UniqueName: \"kubernetes.io/projected/a426988b-e088-4524-8df0-21f725d06d4e-kube-api-access-9xgpm\") pod \"insights-runtime-extractor-ksrjf\" (UID: \"a426988b-e088-4524-8df0-21f725d06d4e\") " pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.145153 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.145048 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ksrjf" Jun 11 16:49:02.281557 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.281528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ksrjf"] Jun 11 16:49:02.285454 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:02.285426 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda426988b_e088_4524_8df0_21f725d06d4e.slice/crio-43b34f7a32b3fe14112c6fa805112e3791538ac6121b0c34fd886aa3b6b17050 WatchSource:0}: Error finding container 43b34f7a32b3fe14112c6fa805112e3791538ac6121b0c34fd886aa3b6b17050: Status 404 returned error can't find the container with id 43b34f7a32b3fe14112c6fa805112e3791538ac6121b0c34fd886aa3b6b17050 Jun 11 16:49:02.444283 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.444200 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2dcgz" Jun 11 16:49:02.469567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.469534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ksrjf" event={"ID":"a426988b-e088-4524-8df0-21f725d06d4e","Type":"ContainerStarted","Data":"35c2436c4f0eadf5d1ea159daedcb609cf9ea3bb1eb533ff50449d3cdad00cde"} Jun 11 16:49:02.469726 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:02.469627 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ksrjf" event={"ID":"a426988b-e088-4524-8df0-21f725d06d4e","Type":"ContainerStarted","Data":"43b34f7a32b3fe14112c6fa805112e3791538ac6121b0c34fd886aa3b6b17050"} Jun 11 16:49:03.110545 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.110511 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:03.113559 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.113541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.116643 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.116620 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 16:49:03.117499 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117464 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 16:49:03.117499 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117494 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 16:49:03.117720 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117511 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 16:49:03.117720 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117555 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 16:49:03.117720 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117572 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 16:49:03.117720 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117630 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fvk45\"" Jun 11 16:49:03.117920 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.117854 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 16:49:03.129285 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.129263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:03.241052 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241022 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.241221 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.241221 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.241221 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241134 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.241221 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241172 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.241221 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.241188 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt7sj\" (UniqueName: \"kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.341979 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.341940 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.341979 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.341977 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt7sj\" (UniqueName: \"kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342000 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342199 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342656 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342866 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342846 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.342930 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.342918 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.344638 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.344616 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.344734 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.344691 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.350887 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.350865 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt7sj\" (UniqueName: \"kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj\") pod \"console-688464486-nfg87\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.422987 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.422948 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:03.478688 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.477700 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ksrjf" event={"ID":"a426988b-e088-4524-8df0-21f725d06d4e","Type":"ContainerStarted","Data":"e11698d2a4e14423dc586b677c6a715830fc63412154afe0f34772b34eb6d5c2"} Jun 11 16:49:03.586051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:03.586017 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:03.590015 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:03.589987 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e069e02_ad4b_4482_abd1_992cbb3af597.slice/crio-63df85d27b11393022eec04839ddb6602c49153149bee482c8ca67bbdb00ad91 WatchSource:0}: Error finding container 63df85d27b11393022eec04839ddb6602c49153149bee482c8ca67bbdb00ad91: Status 404 returned error can't find the container with id 63df85d27b11393022eec04839ddb6602c49153149bee482c8ca67bbdb00ad91 Jun 11 16:49:04.482725 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.482687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688464486-nfg87" event={"ID":"8e069e02-ad4b-4482-abd1-992cbb3af597","Type":"ContainerStarted","Data":"63df85d27b11393022eec04839ddb6602c49153149bee482c8ca67bbdb00ad91"} Jun 11 16:49:04.546946 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.546813 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:49:04.589482 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.547219 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:49:04.589482 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.551771 2557 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 16:49:04.922621 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.922524 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-n5spj"] Jun 11 16:49:04.924770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.924755 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:04.927473 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.927450 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 16:49:04.928428 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.928397 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fgxcz\"" Jun 11 16:49:04.928537 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.928432 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 16:49:04.928537 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.928442 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 11 16:49:04.928537 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.928403 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 16:49:04.928537 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.928499 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 11 16:49:04.935940 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:04.935918 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-n5spj"] Jun 11 16:49:05.057470 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.057432 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.057688 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.057498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9qnl\" (UniqueName: \"kubernetes.io/projected/5ccc1a0d-1568-488f-94a2-cef16964a7d9-kube-api-access-f9qnl\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.057688 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.057565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.057688 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.057651 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1a0d-1568-488f-94a2-cef16964a7d9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.158679 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.158628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9qnl\" (UniqueName: \"kubernetes.io/projected/5ccc1a0d-1568-488f-94a2-cef16964a7d9-kube-api-access-f9qnl\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.158856 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.158727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.158856 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.158761 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1a0d-1568-488f-94a2-cef16964a7d9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.158856 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.158810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.159017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:05.158896 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 11 16:49:05.159017 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:05.158974 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls podName:5ccc1a0d-1568-488f-94a2-cef16964a7d9 nodeName:}" failed. No retries permitted until 2026-06-11 16:49:05.658953296 +0000 UTC m=+301.548084654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-n5spj" (UID: "5ccc1a0d-1568-488f-94a2-cef16964a7d9") : secret "prometheus-operator-tls" not found Jun 11 16:49:05.159492 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.159472 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1a0d-1568-488f-94a2-cef16964a7d9-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.161482 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.161462 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.169934 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.169910 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9qnl\" (UniqueName: \"kubernetes.io/projected/5ccc1a0d-1568-488f-94a2-cef16964a7d9-kube-api-access-f9qnl\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.489336 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.489283 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ksrjf" event={"ID":"a426988b-e088-4524-8df0-21f725d06d4e","Type":"ContainerStarted","Data":"a6b0723191cf323d22a51b652aa5e15b748bd331cc9b356085ad3399280a83f0"} Jun 11 16:49:05.512480 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.512422 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ksrjf" podStartSLOduration=2.323463767 podStartE2EDuration="4.512400869s" totalImagesPullingTime="2.188937102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:49:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:05.510837105 +0000 UTC m=+301.399968483" watchObservedRunningTime="2026-06-11 16:49:05.512400869 +0000 UTC m=+301.401532247" Jun 11 16:49:05.663906 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.663865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.666275 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.666255 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5ccc1a0d-1568-488f-94a2-cef16964a7d9-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-n5spj\" (UID: \"5ccc1a0d-1568-488f-94a2-cef16964a7d9\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:05.833923 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:05.833851 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" Jun 11 16:49:06.959877 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:06.959849 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-n5spj"] Jun 11 16:49:06.965655 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:06.965619 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccc1a0d_1568_488f_94a2_cef16964a7d9.slice/crio-16363366a7609b7890675c3e1ef9976e654b55059ea1abeac5e72ab637092cce WatchSource:0}: Error finding container 16363366a7609b7890675c3e1ef9976e654b55059ea1abeac5e72ab637092cce: Status 404 returned error can't find the container with id 16363366a7609b7890675c3e1ef9976e654b55059ea1abeac5e72ab637092cce Jun 11 16:49:06.968185 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:06.968165 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:49:07.498365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:07.498325 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" event={"ID":"5ccc1a0d-1568-488f-94a2-cef16964a7d9","Type":"ContainerStarted","Data":"16363366a7609b7890675c3e1ef9976e654b55059ea1abeac5e72ab637092cce"} Jun 11 16:49:07.499773 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:07.499746 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688464486-nfg87" event={"ID":"8e069e02-ad4b-4482-abd1-992cbb3af597","Type":"ContainerStarted","Data":"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34"} Jun 11 16:49:07.521455 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:07.521402 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688464486-nfg87" podStartSLOduration=1.250998307 podStartE2EDuration="4.521386372s" totalImagesPullingTime="3.270388065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:07.519534852 +0000 UTC m=+303.408666229" watchObservedRunningTime="2026-06-11 16:49:07.521386372 +0000 UTC m=+303.410517749" Jun 11 16:49:08.505770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:08.505734 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" event={"ID":"5ccc1a0d-1568-488f-94a2-cef16964a7d9","Type":"ContainerStarted","Data":"7d5b9c4ba4f5e6afc60c248100b2c0fdc90fd88d7ce1a3405f51e1a58c24cd35"} Jun 11 16:49:08.506202 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:08.505841 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" event={"ID":"5ccc1a0d-1568-488f-94a2-cef16964a7d9","Type":"ContainerStarted","Data":"d1a1555b514d44673056fa59f1d55ce2a8d7f9c47253d877d0a285901b258ce1"} Jun 11 16:49:08.523648 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:08.523600 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-n5spj" podStartSLOduration=3.099006695 podStartE2EDuration="4.523562723s" totalImagesPullingTime="1.424556028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:08.52271542 +0000 UTC m=+304.411846797" watchObservedRunningTime="2026-06-11 16:49:08.523562723 +0000 UTC m=+304.412694099" Jun 11 16:49:09.316686 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:09.316648 2557 patch_prober.go:28] interesting pod/image-registry-c595d8856-r7dsx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 16:49:09.316851 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:09.316700 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:49:10.312601 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.312549 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jcvws"] Jun 11 16:49:10.314782 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.314766 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.317508 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.317472 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 16:49:10.317654 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.317574 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 16:49:10.317654 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.317599 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nvhnv\"" Jun 11 16:49:10.317768 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.317691 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 16:49:10.506765 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-metrics-client-ca\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.506929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-sys\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.506929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506833 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-textfile\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.506929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-wtmp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.506929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-tls\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.506929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506922 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-root\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.507082 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506938 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.507082 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.506989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp8vp\" (UniqueName: \"kubernetes.io/projected/7f7f016e-b12b-49b2-bb29-ef71237598ab-kube-api-access-wp8vp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.507082 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.507029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608241 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-sys\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608478 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608454 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-textfile\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608664 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-wtmp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-tls\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608801 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-textfile\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608788 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-wtmp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608845 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-root\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608278 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-sys\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608874 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp8vp\" (UniqueName: \"kubernetes.io/projected/7f7f016e-b12b-49b2-bb29-ef71237598ab-kube-api-access-wp8vp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608917 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7f7f016e-b12b-49b2-bb29-ef71237598ab-root\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.608955 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.609253 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.608975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-metrics-client-ca\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.609493 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.609465 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-accelerators-collector-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.609638 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.609618 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f7f016e-b12b-49b2-bb29-ef71237598ab-metrics-client-ca\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.611388 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.611366 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-tls\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.611704 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.611687 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f7f016e-b12b-49b2-bb29-ef71237598ab-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.617437 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.617414 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp8vp\" (UniqueName: \"kubernetes.io/projected/7f7f016e-b12b-49b2-bb29-ef71237598ab-kube-api-access-wp8vp\") pod \"node-exporter-jcvws\" (UID: \"7f7f016e-b12b-49b2-bb29-ef71237598ab\") " pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.624238 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:10.624219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jcvws" Jun 11 16:49:10.636561 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:10.636539 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7f016e_b12b_49b2_bb29_ef71237598ab.slice/crio-fdd2900605e83cdf0bfe769eef8aea385d41876f71ac9470d134a1766907ea8a WatchSource:0}: Error finding container fdd2900605e83cdf0bfe769eef8aea385d41876f71ac9470d134a1766907ea8a: Status 404 returned error can't find the container with id fdd2900605e83cdf0bfe769eef8aea385d41876f71ac9470d134a1766907ea8a Jun 11 16:49:11.421696 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.421667 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:49:11.425318 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.425301 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.428616 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.428595 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 16:49:11.428945 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.428919 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 16:49:11.429486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429468 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 16:49:11.429564 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429527 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 16:49:11.429564 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429552 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 16:49:11.429691 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429607 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 16:49:11.429691 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429632 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 16:49:11.429915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429698 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v99mc\"" Jun 11 16:49:11.429915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.429816 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 16:49:11.436995 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.436974 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 16:49:11.452257 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.452232 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:49:11.518319 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.518290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcvws" event={"ID":"7f7f016e-b12b-49b2-bb29-ef71237598ab","Type":"ContainerStarted","Data":"f8d4f4e97fa8c52ac4935bc59d043edb9d3548bbd5a1bf32e3672a17a4b6a8a0"} Jun 11 16:49:11.518474 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.518341 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcvws" event={"ID":"7f7f016e-b12b-49b2-bb29-ef71237598ab","Type":"ContainerStarted","Data":"fdd2900605e83cdf0bfe769eef8aea385d41876f71ac9470d134a1766907ea8a"} Jun 11 16:49:11.616994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.616971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617093 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617001 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617093 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617028 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617093 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617093 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617079 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617176 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617203 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617230 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617277 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617302 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd47m\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.617421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.617319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718349 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718315 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718349 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718618 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718618 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718551 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718618 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718797 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718797 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718722 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718797 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718797 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718780 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.718990 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:11.718853 2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 11 16:49:11.718990 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:11.718915 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls podName:05d67312-99e6-4bb1-ab05-8d96e026199a nodeName:}" failed. No retries permitted until 2026-06-11 16:49:12.218894894 +0000 UTC m=+308.108026269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a") : secret "alertmanager-main-tls" not found Jun 11 16:49:11.718990 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.718978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.719157 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.719013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.719157 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.719053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd47m\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.719157 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.719095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.720457 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.719342 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.720457 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.719993 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.720457 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.720180 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.722322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.721960 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.722322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.721979 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.722322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.722074 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.722322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.722260 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.722857 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.722739 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.728110 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.723172 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.728110 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.724802 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.728110 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.724978 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:11.730737 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:11.730716 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd47m\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:12.222381 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.222343 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:12.224753 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.224724 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:12.360461 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.360408 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:49:12.444405 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.444383 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:49:12.503643 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.503550 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:49:12.506753 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:12.506715 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d67312_99e6_4bb1_ab05_8d96e026199a.slice/crio-924e73dea231d1c0f8b463add8073e516a94ce98f37326e04e9e7daa7f802b0b WatchSource:0}: Error finding container 924e73dea231d1c0f8b463add8073e516a94ce98f37326e04e9e7daa7f802b0b: Status 404 returned error can't find the container with id 924e73dea231d1c0f8b463add8073e516a94ce98f37326e04e9e7daa7f802b0b Jun 11 16:49:12.523003 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.522973 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"924e73dea231d1c0f8b463add8073e516a94ce98f37326e04e9e7daa7f802b0b"} Jun 11 16:49:12.524322 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.524296 2557 generic.go:356] "Generic (PLEG): container finished" podID="7f7f016e-b12b-49b2-bb29-ef71237598ab" containerID="f8d4f4e97fa8c52ac4935bc59d043edb9d3548bbd5a1bf32e3672a17a4b6a8a0" exitCode=0 Jun 11 16:49:12.524449 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:12.524345 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcvws" event={"ID":"7f7f016e-b12b-49b2-bb29-ef71237598ab","Type":"ContainerDied","Data":"f8d4f4e97fa8c52ac4935bc59d043edb9d3548bbd5a1bf32e3672a17a4b6a8a0"} Jun 11 16:49:13.424009 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.423966 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:13.424185 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.424042 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:13.429400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.429372 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:13.511857 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.511824 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:49:13.531939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.531903 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcvws" event={"ID":"7f7f016e-b12b-49b2-bb29-ef71237598ab","Type":"ContainerStarted","Data":"d467d7295342e2c51895fdf843fa2f965ddd8f03738ac3a6b0cf6fb7e08d593f"} Jun 11 16:49:13.532135 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.531970 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jcvws" event={"ID":"7f7f016e-b12b-49b2-bb29-ef71237598ab","Type":"ContainerStarted","Data":"8844a79653e4bc118af8ade6862609d6f4768461e14d2a2f086eb8eb5a9712f3"} Jun 11 16:49:13.536007 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.535985 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:13.553269 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:13.553216 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jcvws" podStartSLOduration=2.710136832 podStartE2EDuration="3.553200818s" totalImagesPullingTime="772.936556ms" totalInitContainerRuntime="70.12743ms" isStatefulPod=true podCreationTimestamp="2026-06-11 16:49:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:13.552478187 +0000 UTC m=+309.441609576" watchObservedRunningTime="2026-06-11 16:49:13.553200818 +0000 UTC m=+309.442332249" Jun 11 16:49:14.535994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.535956 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5" exitCode=0 Jun 11 16:49:14.536471 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.536039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5"} Jun 11 16:49:14.711834 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.711801 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b6c985589-kghpg"] Jun 11 16:49:14.714106 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.714090 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.716833 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.716806 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 11 16:49:14.717075 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.717060 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 16:49:14.717192 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.717175 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 11 16:49:14.717241 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.717204 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 11 16:49:14.717241 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.717230 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4bgsc\"" Jun 11 16:49:14.717349 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.717301 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3bfa9ejjgvg7h\"" Jun 11 16:49:14.728562 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.728539 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b6c985589-kghpg"] Jun 11 16:49:14.743547 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-tls\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743690 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743564 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743690 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743648 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-metrics-server-audit-profiles\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743778 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-client-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743778 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743741 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-client-certs\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743841 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dd90dad-f793-4a32-8011-a963805fd8b6-audit-log\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.743841 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.743802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4hjk\" (UniqueName: \"kubernetes.io/projected/4dd90dad-f793-4a32-8011-a963805fd8b6-kube-api-access-r4hjk\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.844935 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844849 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.844935 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-metrics-server-audit-profiles\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.844935 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-client-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844953 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-client-certs\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dd90dad-f793-4a32-8011-a963805fd8b6-audit-log\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.844993 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4hjk\" (UniqueName: \"kubernetes.io/projected/4dd90dad-f793-4a32-8011-a963805fd8b6-kube-api-access-r4hjk\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845336 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.845241 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-tls\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845421 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.845395 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dd90dad-f793-4a32-8011-a963805fd8b6-audit-log\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.845759 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.845730 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.846015 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.845996 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dd90dad-f793-4a32-8011-a963805fd8b6-metrics-server-audit-profiles\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.847573 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.847550 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-tls\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.847752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.847733 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-client-ca-bundle\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.847790 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.847755 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dd90dad-f793-4a32-8011-a963805fd8b6-secret-metrics-server-client-certs\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:14.865378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:14.865349 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4hjk\" (UniqueName: \"kubernetes.io/projected/4dd90dad-f793-4a32-8011-a963805fd8b6-kube-api-access-r4hjk\") pod \"metrics-server-7b6c985589-kghpg\" (UID: \"4dd90dad-f793-4a32-8011-a963805fd8b6\") " pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:15.023870 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.023569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:15.077852 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.077816 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-f57fv"] Jun 11 16:49:15.080852 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.080831 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.083602 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.083546 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rwvxt\"" Jun 11 16:49:15.084262 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.084199 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 11 16:49:15.090741 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.090621 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-f57fv"] Jun 11 16:49:15.152811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.148853 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhvs4\" (UniqueName: \"kubernetes.io/projected/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-kube-api-access-xhvs4\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.152811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.149064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.189327 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.189295 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b6c985589-kghpg"] Jun 11 16:49:15.193769 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:15.193501 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd90dad_f793_4a32_8011_a963805fd8b6.slice/crio-6fa0bf8a8789873bb9eba11b66771b720db013d5b22e1ce650db933b241a95c6 WatchSource:0}: Error finding container 6fa0bf8a8789873bb9eba11b66771b720db013d5b22e1ce650db933b241a95c6: Status 404 returned error can't find the container with id 6fa0bf8a8789873bb9eba11b66771b720db013d5b22e1ce650db933b241a95c6 Jun 11 16:49:15.250331 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.250296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.250483 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.250395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhvs4\" (UniqueName: \"kubernetes.io/projected/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-kube-api-access-xhvs4\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.250483 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:15.250454 2557 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 16:49:15.250549 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:15.250524 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert podName:c81680e2-bdc4-4ffe-8f52-3b4047dcc88e nodeName:}" failed. No retries permitted until 2026-06-11 16:49:15.750506622 +0000 UTC m=+311.639637990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-f57fv" (UID: "c81680e2-bdc4-4ffe-8f52-3b4047dcc88e") : secret "monitoring-plugin-cert" not found Jun 11 16:49:15.270895 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.270865 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhvs4\" (UniqueName: \"kubernetes.io/projected/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-kube-api-access-xhvs4\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.542028 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.541994 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" event={"ID":"4dd90dad-f793-4a32-8011-a963805fd8b6","Type":"ContainerStarted","Data":"6fa0bf8a8789873bb9eba11b66771b720db013d5b22e1ce650db933b241a95c6"} Jun 11 16:49:15.617699 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.617668 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-fc455c9bf-thmmj"] Jun 11 16:49:15.621156 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.621138 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.625121 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.625095 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 11 16:49:15.625276 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.625257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 11 16:49:15.625760 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.625739 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 11 16:49:15.625944 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.625925 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-lhs5d\"" Jun 11 16:49:15.627042 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.627022 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 11 16:49:15.628389 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.628362 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 11 16:49:15.635903 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.635883 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 11 16:49:15.654784 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.654740 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fc455c9bf-thmmj"] Jun 11 16:49:15.655158 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-metrics-client-ca\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655187 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655363 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655272 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655363 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-serving-certs-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655363 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655415 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-federate-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.655512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.655498 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbbvg\" (UniqueName: \"kubernetes.io/projected/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-kube-api-access-kbbvg\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756349 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756308 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbbvg\" (UniqueName: \"kubernetes.io/projected/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-kube-api-access-kbbvg\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-metrics-client-ca\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756470 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756617 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-serving-certs-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756787 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756708 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.756787 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.757541 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.756911 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-federate-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.757541 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.757371 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-trusted-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.757541 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.757381 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-metrics-client-ca\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.757541 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.757514 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-serving-certs-ca-bundle\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.759543 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.759497 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.759805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.759783 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.760298 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.760276 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c81680e2-bdc4-4ffe-8f52-3b4047dcc88e-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-f57fv\" (UID: \"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:15.760749 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.760731 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-federate-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.761008 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.760989 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-telemeter-client-tls\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.770085 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.770034 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbbvg\" (UniqueName: \"kubernetes.io/projected/cdfa4b33-c4f3-4fad-9db5-26939cb0a094-kube-api-access-kbbvg\") pod \"telemeter-client-fc455c9bf-thmmj\" (UID: \"cdfa4b33-c4f3-4fad-9db5-26939cb0a094\") " pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.931102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.930785 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" Jun 11 16:49:15.996792 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:15.996751 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:16.271592 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.271495 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-f57fv"] Jun 11 16:49:16.273040 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:16.272998 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81680e2_bdc4_4ffe_8f52_3b4047dcc88e.slice/crio-ee9865abac6d3a93399c383bd3dd616686a844fca233e2e53d251df98d6d7167 WatchSource:0}: Error finding container ee9865abac6d3a93399c383bd3dd616686a844fca233e2e53d251df98d6d7167: Status 404 returned error can't find the container with id ee9865abac6d3a93399c383bd3dd616686a844fca233e2e53d251df98d6d7167 Jun 11 16:49:16.298863 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.298835 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-fc455c9bf-thmmj"] Jun 11 16:49:16.302903 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:49:16.302875 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfa4b33_c4f3_4fad_9db5_26939cb0a094.slice/crio-9930236d429f7f655c75843b81b85b2852b96970ec671515eacb142c5d957ae8 WatchSource:0}: Error finding container 9930236d429f7f655c75843b81b85b2852b96970ec671515eacb142c5d957ae8: Status 404 returned error can't find the container with id 9930236d429f7f655c75843b81b85b2852b96970ec671515eacb142c5d957ae8 Jun 11 16:49:16.551463 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.551416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5"} Jun 11 16:49:16.551965 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.551882 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6"} Jun 11 16:49:16.551965 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.551950 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de"} Jun 11 16:49:16.553212 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.553188 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" event={"ID":"cdfa4b33-c4f3-4fad-9db5-26939cb0a094","Type":"ContainerStarted","Data":"9930236d429f7f655c75843b81b85b2852b96970ec671515eacb142c5d957ae8"} Jun 11 16:49:16.554736 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.554708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" event={"ID":"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e","Type":"ContainerStarted","Data":"ee9865abac6d3a93399c383bd3dd616686a844fca233e2e53d251df98d6d7167"} Jun 11 16:49:16.763355 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.763317 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:49:16.769481 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.769447 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.776607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776414 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 16:49:16.776607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776456 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 16:49:16.776607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776420 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 16:49:16.776607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776483 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 16:49:16.776914 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776626 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 16:49:16.776969 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776940 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3is6ea7unc174\"" Jun 11 16:49:16.777016 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.776996 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 16:49:16.777074 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777032 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 16:49:16.777122 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 16:49:16.777231 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777182 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 16:49:16.777357 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777269 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 16:49:16.777357 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777314 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 16:49:16.777477 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777383 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sp6sw\"" Jun 11 16:49:16.777822 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.777783 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 16:49:16.781223 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.781202 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 16:49:16.781859 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.781836 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 16:49:16.789510 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.789484 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:49:16.867710 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867682 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.867854 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867716 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.867854 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.867854 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867763 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.867854 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867876 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867898 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.867985 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868069 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868114 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thmrz\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868173 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868205 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868230 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868321 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.868668 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.868381 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969219 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969393 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmrz\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969484 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969639 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969667 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.971735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969702 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972548 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969727 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972548 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972548 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.969790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972548 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.970552 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972548 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.971499 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.972813 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.972742 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.976118 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.974790 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.978598 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.976516 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.979481 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.978927 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.983790 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.983764 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.984596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.984376 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.984596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.984533 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.984750 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.984729 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.985702 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.985657 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.986880 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.986854 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.988090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.988029 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.988431 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.988406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.989108 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.989044 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.989832 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.989674 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.990535 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.990473 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmrz\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.990631 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.990555 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.991177 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.991117 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:16.992498 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:16.992457 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:17.083919 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.083859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:17.283269 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.282932 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:49:17.571607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.570675 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" exitCode=0 Jun 11 16:49:17.571607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.570791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} Jun 11 16:49:17.571607 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.570908 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"fdce7a2234bbf2d88e0a537e8bded6693501f77e34906b588174462d9ae00be5"} Jun 11 16:49:17.577622 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.577423 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" event={"ID":"4dd90dad-f793-4a32-8011-a963805fd8b6","Type":"ContainerStarted","Data":"e4fbd00403cef7efa8fbf6ae2445f2d030855bc3291b2c4f5d74a49f640a39e5"} Jun 11 16:49:17.591166 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.591041 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a"} Jun 11 16:49:17.591166 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.591119 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885"} Jun 11 16:49:17.635208 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:17.635100 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" podStartSLOduration=1.939562709 podStartE2EDuration="3.635075079s" totalImagesPullingTime="1.69551237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:17.63146396 +0000 UTC m=+313.520595336" watchObservedRunningTime="2026-06-11 16:49:17.635075079 +0000 UTC m=+313.524206461" Jun 11 16:49:18.598703 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.598568 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" event={"ID":"c81680e2-bdc4-4ffe-8f52-3b4047dcc88e","Type":"ContainerStarted","Data":"4716b816316d92a24eba0b13b568d3a1dcd9f69df763185e4f4502ca1d6af2a2"} Jun 11 16:49:18.598703 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.598662 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:18.605683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.605653 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" Jun 11 16:49:18.606074 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.605993 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerStarted","Data":"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f"} Jun 11 16:49:18.617300 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.617231 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-f57fv" podStartSLOduration=1.676318268 podStartE2EDuration="3.617210922s" totalImagesPullingTime="1.940892654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:18.616747953 +0000 UTC m=+314.505879331" watchObservedRunningTime="2026-06-11 16:49:18.617210922 +0000 UTC m=+314.506342300" Jun 11 16:49:18.651147 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:18.650451 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.763361051 podStartE2EDuration="7.650431283s" totalImagesPullingTime="3.871632863s" totalInitContainerRuntime="15.437369ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:18.648307851 +0000 UTC m=+314.537439231" watchObservedRunningTime="2026-06-11 16:49:18.650431283 +0000 UTC m=+314.539562661" Jun 11 16:49:19.612561 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:19.612523 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" event={"ID":"cdfa4b33-c4f3-4fad-9db5-26939cb0a094","Type":"ContainerStarted","Data":"f585f2288693ac87290fd07573c05eb4c4bb364e3b01d34e6244e8de2de218a5"} Jun 11 16:49:19.612994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:19.612609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" event={"ID":"cdfa4b33-c4f3-4fad-9db5-26939cb0a094","Type":"ContainerStarted","Data":"ab6c72c27d024cbf81bdc6047b74e0ccfa8dd4e7d80d1a53abe47982efeb0cb7"} Jun 11 16:49:19.612994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:19.612641 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" event={"ID":"cdfa4b33-c4f3-4fad-9db5-26939cb0a094","Type":"ContainerStarted","Data":"276c14734670eaa5d70f95895fd75e5fb77b4c03e8c323c306714939d6dc95e7"} Jun 11 16:49:19.636907 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:19.636843 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-fc455c9bf-thmmj" podStartSLOduration=1.7290244750000001 podStartE2EDuration="4.636820549s" totalImagesPullingTime="2.907796074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:19.635013063 +0000 UTC m=+315.524144440" watchObservedRunningTime="2026-06-11 16:49:19.636820549 +0000 UTC m=+315.525951928" Jun 11 16:49:21.628328 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:21.628284 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} Jun 11 16:49:21.628842 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:21.628383 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} Jun 11 16:49:23.640395 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:23.640359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} Jun 11 16:49:23.640793 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:23.640436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} Jun 11 16:49:23.640793 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:23.640479 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} Jun 11 16:49:23.640793 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:23.640523 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerStarted","Data":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} Jun 11 16:49:23.672839 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:23.672769 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.474254293 podStartE2EDuration="7.672745014s" totalImagesPullingTime="5.169251787s" totalInitContainerRuntime="29.238934ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:49:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:49:23.669449436 +0000 UTC m=+319.558580812" watchObservedRunningTime="2026-06-11 16:49:23.672745014 +0000 UTC m=+319.561876392" Jun 11 16:49:24.179901 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:24.179868 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:49:27.084174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:27.084139 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:49:31.348083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:31.348053 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:35.024653 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:35.024608 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:35.025021 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:35.024678 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:48.743317 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:48.743280 2557 generic.go:356] "Generic (PLEG): container finished" podID="6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11" containerID="db42c4be8ba0761a3aea975360e365c540ce88535b5664807e2eab876aac06d8" exitCode=0 Jun 11 16:49:48.743725 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:48.743329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" event={"ID":"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11","Type":"ContainerDied","Data":"db42c4be8ba0761a3aea975360e365c540ce88535b5664807e2eab876aac06d8"} Jun 11 16:49:48.743807 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:48.743788 2557 scope.go:117] "RemoveContainer" containerID="db42c4be8ba0761a3aea975360e365c540ce88535b5664807e2eab876aac06d8" Jun 11 16:49:49.208784 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.208732 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" containerName="registry" containerID="cri-o://5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca" gracePeriod=30 Jun 11 16:49:49.450329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.450305 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:49:49.495359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495332 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvk2t\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495377 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495420 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495455 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495479 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495531 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495513 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495800 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495610 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.495800 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495649 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates\") pod \"9de10675-cbff-4637-93ef-a69149df0c7b\" (UID: \"9de10675-cbff-4637-93ef-a69149df0c7b\") " Jun 11 16:49:49.496008 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.495977 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:49:49.496315 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.496288 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:49:49.498002 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.497946 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t" (OuterVolumeSpecName: "kube-api-access-tvk2t") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "kube-api-access-tvk2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:49:49.498152 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.498116 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:49:49.498152 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.498128 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:49:49.498290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.498270 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:49:49.498378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.498352 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:49:49.503336 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.503314 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9de10675-cbff-4637-93ef-a69149df0c7b" (UID: "9de10675-cbff-4637-93ef-a69149df0c7b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:49:49.597006 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.596950 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvk2t\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-kube-api-access-tvk2t\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597006 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597006 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-bound-sa-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597006 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597021 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9de10675-cbff-4637-93ef-a69149df0c7b-registry-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597034 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9de10675-cbff-4637-93ef-a69149df0c7b-ca-trust-extracted\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597047 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-trusted-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597061 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-image-registry-private-configuration\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597073 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9de10675-cbff-4637-93ef-a69149df0c7b-installation-pull-secrets\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.597245 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.597086 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9de10675-cbff-4637-93ef-a69149df0c7b-registry-certificates\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:49.749279 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.749194 2557 generic.go:356] "Generic (PLEG): container finished" podID="9de10675-cbff-4637-93ef-a69149df0c7b" containerID="5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca" exitCode=0 Jun 11 16:49:49.749279 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.749242 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" Jun 11 16:49:49.749808 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.749282 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" event={"ID":"9de10675-cbff-4637-93ef-a69149df0c7b","Type":"ContainerDied","Data":"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca"} Jun 11 16:49:49.749808 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.749338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c595d8856-r7dsx" event={"ID":"9de10675-cbff-4637-93ef-a69149df0c7b","Type":"ContainerDied","Data":"c2770ab6f5f68099655c7b2108dd8cf5d7bd0c629e51a49225b20971a2ccde1f"} Jun 11 16:49:49.749808 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.749368 2557 scope.go:117] "RemoveContainer" containerID="5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca" Jun 11 16:49:49.751343 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.751322 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-r8gzr" event={"ID":"6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11","Type":"ContainerStarted","Data":"c1ed7e91882f83d1ea24d69441dbed909e70b1679006baeb96c3718e7ae93705"} Jun 11 16:49:49.758108 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.758091 2557 scope.go:117] "RemoveContainer" containerID="5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca" Jun 11 16:49:49.758351 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:49.758331 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca\": container with ID starting with 5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca not found: ID does not exist" containerID="5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca" Jun 11 16:49:49.758418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.758365 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca"} err="failed to get container status \"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca\": rpc error: code = NotFound desc = could not find container \"5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca\": container with ID starting with 5facf5d30e8231216524b5efa68051743420a83ad22ae77a206d571cc8a966ca not found: ID does not exist" Jun 11 16:49:49.785926 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.785896 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:49:49.790485 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:49.790460 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c595d8856-r7dsx"] Jun 11 16:49:50.517594 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:50.517551 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" path="/var/lib/kubelet/pods/9de10675-cbff-4637-93ef-a69149df0c7b/volumes" Jun 11 16:49:51.131734 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:51.131708 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log" Jun 11 16:49:51.155121 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:51.155096 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwwn2_58ac9bf0-ee3c-4e93-a17d-b78f099d991f/serve-healthcheck-canary/0.log" Jun 11 16:49:55.029865 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:55.029832 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:55.033802 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:55.033777 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b6c985589-kghpg" Jun 11 16:49:56.368623 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.368561 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-688464486-nfg87" podUID="8e069e02-ad4b-4482-abd1-992cbb3af597" containerName="console" containerID="cri-o://bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34" gracePeriod=15 Jun 11 16:49:56.619056 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.618998 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:56.664484 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664453 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.664644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664516 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.664644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664547 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.664644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664573 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.664878 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664675 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.664878 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664706 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tt7sj\" (UniqueName: \"kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj\") pod \"8e069e02-ad4b-4482-abd1-992cbb3af597\" (UID: \"8e069e02-ad4b-4482-abd1-992cbb3af597\") " Jun 11 16:49:56.665009 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664988 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:49:56.665123 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.664998 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca" (OuterVolumeSpecName: "service-ca") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:49:56.665257 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.665171 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config" (OuterVolumeSpecName: "console-config") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:49:56.667022 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.666997 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:49:56.667213 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.667176 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:49:56.667291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.667237 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj" (OuterVolumeSpecName: "kube-api-access-tt7sj") pod "8e069e02-ad4b-4482-abd1-992cbb3af597" (UID: "8e069e02-ad4b-4482-abd1-992cbb3af597"). InnerVolumeSpecName "kube-api-access-tt7sj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:49:56.766039 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766007 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-oauth-serving-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.766039 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766035 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-service-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.766039 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766046 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-serving-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.766255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766055 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e069e02-ad4b-4482-abd1-992cbb3af597-console-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.766255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766063 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt7sj\" (UniqueName: \"kubernetes.io/projected/8e069e02-ad4b-4482-abd1-992cbb3af597-kube-api-access-tt7sj\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.766255 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.766073 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e069e02-ad4b-4482-abd1-992cbb3af597-console-oauth-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:49:56.783017 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.782988 2557 generic.go:356] "Generic (PLEG): container finished" podID="8e069e02-ad4b-4482-abd1-992cbb3af597" containerID="bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34" exitCode=0 Jun 11 16:49:56.783174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.783027 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688464486-nfg87" Jun 11 16:49:56.783174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.783080 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688464486-nfg87" event={"ID":"8e069e02-ad4b-4482-abd1-992cbb3af597","Type":"ContainerDied","Data":"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34"} Jun 11 16:49:56.783174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.783121 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688464486-nfg87" event={"ID":"8e069e02-ad4b-4482-abd1-992cbb3af597","Type":"ContainerDied","Data":"63df85d27b11393022eec04839ddb6602c49153149bee482c8ca67bbdb00ad91"} Jun 11 16:49:56.783174 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.783140 2557 scope.go:117] "RemoveContainer" containerID="bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34" Jun 11 16:49:56.791690 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.791668 2557 scope.go:117] "RemoveContainer" containerID="bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34" Jun 11 16:49:56.791944 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:49:56.791923 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34\": container with ID starting with bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34 not found: ID does not exist" containerID="bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34" Jun 11 16:49:56.791996 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.791952 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34"} err="failed to get container status \"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34\": rpc error: code = NotFound desc = could not find container \"bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34\": container with ID starting with bd77cffcdf78ad0e5fa14017497967b5a48d92ba242ab4b3fa97fb4afa457c34 not found: ID does not exist" Jun 11 16:49:56.807004 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.806980 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:56.812234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:56.812213 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688464486-nfg87"] Jun 11 16:49:58.517481 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:49:58.517433 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e069e02-ad4b-4482-abd1-992cbb3af597" path="/var/lib/kubelet/pods/8e069e02-ad4b-4482-abd1-992cbb3af597/volumes" Jun 11 16:50:17.084883 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.084847 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:17.117951 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.117922 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:17.558114 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.558077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:50:17.560429 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.560406 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b05df1bd-4184-445d-be31-e2364f0f5079-metrics-certs\") pod \"network-metrics-daemon-k8t85\" (UID: \"b05df1bd-4184-445d-be31-e2364f0f5079\") " pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:50:17.716117 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.716084 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bmxfr\"" Jun 11 16:50:17.723338 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.723314 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k8t85" Jun 11 16:50:17.853070 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.853047 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k8t85"] Jun 11 16:50:17.855519 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:50:17.855486 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb05df1bd_4184_445d_be31_e2364f0f5079.slice/crio-81202e84f99d2db6142f8724deed49e72d2d660c9764b3ab574473aef1f2986b WatchSource:0}: Error finding container 81202e84f99d2db6142f8724deed49e72d2d660c9764b3ab574473aef1f2986b: Status 404 returned error can't find the container with id 81202e84f99d2db6142f8724deed49e72d2d660c9764b3ab574473aef1f2986b Jun 11 16:50:17.870353 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.870329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8t85" event={"ID":"b05df1bd-4184-445d-be31-e2364f0f5079","Type":"ContainerStarted","Data":"81202e84f99d2db6142f8724deed49e72d2d660c9764b3ab574473aef1f2986b"} Jun 11 16:50:17.885700 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:17.885679 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:18.876516 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:18.876478 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8t85" event={"ID":"b05df1bd-4184-445d-be31-e2364f0f5079","Type":"ContainerStarted","Data":"15969d157c4c7cd9353c5b042270a8b4e0404f5ba1ec5203227da3de5e1ffec4"} Jun 11 16:50:18.876916 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:18.876568 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k8t85" event={"ID":"b05df1bd-4184-445d-be31-e2364f0f5079","Type":"ContainerStarted","Data":"4ea8293c20b309261079ff1b60efacb7b87c50fd61bb051d704f69e47f19eb50"} Jun 11 16:50:18.895084 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:18.895012 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k8t85" podStartSLOduration=374.105646547 podStartE2EDuration="6m14.89499656s" totalImagesPullingTime="789.350013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:50:18.893152329 +0000 UTC m=+374.782283709" watchObservedRunningTime="2026-06-11 16:50:18.89499656 +0000 UTC m=+374.784127936" Jun 11 16:50:30.837902 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.837864 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:30.838302 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.838214 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="alertmanager" containerID="cri-o://c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" gracePeriod=120 Jun 11 16:50:30.838302 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.838249 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-metric" containerID="cri-o://4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" gracePeriod=120 Jun 11 16:50:30.838422 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.838289 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-web" containerID="cri-o://6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" gracePeriod=120 Jun 11 16:50:30.838422 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.838305 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy" containerID="cri-o://8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" gracePeriod=120 Jun 11 16:50:30.839568 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.839158 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="prom-label-proxy" containerID="cri-o://1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" gracePeriod=120 Jun 11 16:50:30.839669 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:30.839517 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="config-reloader" containerID="cri-o://a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" gracePeriod=120 Jun 11 16:50:31.933278 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933249 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" exitCode=0 Jun 11 16:50:31.933278 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933275 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" exitCode=0 Jun 11 16:50:31.933278 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933284 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" exitCode=0 Jun 11 16:50:31.933689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933295 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" exitCode=0 Jun 11 16:50:31.933689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f"} Jun 11 16:50:31.933689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933408 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885"} Jun 11 16:50:31.933689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6"} Jun 11 16:50:31.933689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:31.933459 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de"} Jun 11 16:50:32.099691 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.099663 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:32.176443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176356 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176391 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176416 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176437 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd47m\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176794 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176458 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176794 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176498 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176794 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176524 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.176930 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.176840 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177127 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177177 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177231 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177257 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177326 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177371 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets\") pod \"05d67312-99e6-4bb1-ab05-8d96e026199a\" (UID: \"05d67312-99e6-4bb1-ab05-8d96e026199a\") " Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.177672 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-metrics-client-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.178026 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:32.178320 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.178280 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:50:32.179436 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.179400 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m" (OuterVolumeSpecName: "kube-api-access-xd47m") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "kube-api-access-xd47m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:50:32.179744 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.179716 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.180824 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.180793 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume" (OuterVolumeSpecName: "config-volume") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.181092 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.181061 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.181195 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.181101 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.181307 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.181276 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out" (OuterVolumeSpecName: "config-out") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:50:32.181863 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.181833 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:50:32.181998 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.181975 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.184378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.184357 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.192031 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.192011 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config" (OuterVolumeSpecName: "web-config") pod "05d67312-99e6-4bb1-ab05-8d96e026199a" (UID: "05d67312-99e6-4bb1-ab05-8d96e026199a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:32.278748 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278710 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-main-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278748 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278740 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278748 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278750 2557 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-config-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278759 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278770 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-web-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278778 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-config-out\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278786 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/05d67312-99e6-4bb1-ab05-8d96e026199a-alertmanager-main-db\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278794 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278803 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-tls-assets\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278811 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278819 2557 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/05d67312-99e6-4bb1-ab05-8d96e026199a-cluster-tls-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.278963 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.278828 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd47m\" (UniqueName: \"kubernetes.io/projected/05d67312-99e6-4bb1-ab05-8d96e026199a-kube-api-access-xd47m\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:32.940762 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940724 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" exitCode=0 Jun 11 16:50:32.940762 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940757 2557 generic.go:356] "Generic (PLEG): container finished" podID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerID="6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" exitCode=0 Jun 11 16:50:32.941187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a"} Jun 11 16:50:32.941187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940815 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:32.941187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940859 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5"} Jun 11 16:50:32.941187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940885 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"05d67312-99e6-4bb1-ab05-8d96e026199a","Type":"ContainerDied","Data":"924e73dea231d1c0f8b463add8073e516a94ce98f37326e04e9e7daa7f802b0b"} Jun 11 16:50:32.941187 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.940921 2557 scope.go:117] "RemoveContainer" containerID="1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" Jun 11 16:50:32.948544 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.948527 2557 scope.go:117] "RemoveContainer" containerID="4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" Jun 11 16:50:32.955721 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.955702 2557 scope.go:117] "RemoveContainer" containerID="8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" Jun 11 16:50:32.962638 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.962446 2557 scope.go:117] "RemoveContainer" containerID="6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" Jun 11 16:50:32.964635 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.964614 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:32.969731 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.969710 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:32.970789 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.970774 2557 scope.go:117] "RemoveContainer" containerID="a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" Jun 11 16:50:32.977919 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.977902 2557 scope.go:117] "RemoveContainer" containerID="c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" Jun 11 16:50:32.984911 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.984893 2557 scope.go:117] "RemoveContainer" containerID="84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5" Jun 11 16:50:32.991621 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.991600 2557 scope.go:117] "RemoveContainer" containerID="1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" Jun 11 16:50:32.991873 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.991853 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f\": container with ID starting with 1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f not found: ID does not exist" containerID="1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" Jun 11 16:50:32.991951 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.991886 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f"} err="failed to get container status \"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f\": rpc error: code = NotFound desc = could not find container \"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f\": container with ID starting with 1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f not found: ID does not exist" Jun 11 16:50:32.991951 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.991913 2557 scope.go:117] "RemoveContainer" containerID="4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" Jun 11 16:50:32.992263 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.992228 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a\": container with ID starting with 4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a not found: ID does not exist" containerID="4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" Jun 11 16:50:32.992366 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.992274 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a"} err="failed to get container status \"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a\": rpc error: code = NotFound desc = could not find container \"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a\": container with ID starting with 4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a not found: ID does not exist" Jun 11 16:50:32.992366 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.992303 2557 scope.go:117] "RemoveContainer" containerID="8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" Jun 11 16:50:32.992724 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.992674 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885\": container with ID starting with 8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885 not found: ID does not exist" containerID="8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" Jun 11 16:50:32.992724 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.992706 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885"} err="failed to get container status \"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885\": rpc error: code = NotFound desc = could not find container \"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885\": container with ID starting with 8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885 not found: ID does not exist" Jun 11 16:50:32.992724 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.992728 2557 scope.go:117] "RemoveContainer" containerID="6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" Jun 11 16:50:32.992987 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.992971 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5\": container with ID starting with 6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5 not found: ID does not exist" containerID="6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" Jun 11 16:50:32.993032 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.992992 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5"} err="failed to get container status \"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5\": rpc error: code = NotFound desc = could not find container \"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5\": container with ID starting with 6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5 not found: ID does not exist" Jun 11 16:50:32.993032 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993006 2557 scope.go:117] "RemoveContainer" containerID="a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" Jun 11 16:50:32.993263 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.993244 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6\": container with ID starting with a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6 not found: ID does not exist" containerID="a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" Jun 11 16:50:32.993316 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993270 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6"} err="failed to get container status \"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6\": rpc error: code = NotFound desc = could not find container \"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6\": container with ID starting with a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6 not found: ID does not exist" Jun 11 16:50:32.993316 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993290 2557 scope.go:117] "RemoveContainer" containerID="c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" Jun 11 16:50:32.993560 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.993545 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de\": container with ID starting with c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de not found: ID does not exist" containerID="c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" Jun 11 16:50:32.993670 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993562 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de"} err="failed to get container status \"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de\": rpc error: code = NotFound desc = could not find container \"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de\": container with ID starting with c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de not found: ID does not exist" Jun 11 16:50:32.993670 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993574 2557 scope.go:117] "RemoveContainer" containerID="84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5" Jun 11 16:50:32.993838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:32.993825 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5\": container with ID starting with 84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5 not found: ID does not exist" containerID="84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5" Jun 11 16:50:32.993881 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993841 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5"} err="failed to get container status \"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5\": rpc error: code = NotFound desc = could not find container \"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5\": container with ID starting with 84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5 not found: ID does not exist" Jun 11 16:50:32.993881 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.993852 2557 scope.go:117] "RemoveContainer" containerID="1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f" Jun 11 16:50:32.994044 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994027 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f"} err="failed to get container status \"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f\": rpc error: code = NotFound desc = could not find container \"1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f\": container with ID starting with 1ea7b9e633292e8e8904ecd2de586194ca55f7c76fb22fb24bc5a26f508ef92f not found: ID does not exist" Jun 11 16:50:32.994082 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994045 2557 scope.go:117] "RemoveContainer" containerID="4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a" Jun 11 16:50:32.994254 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994233 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a"} err="failed to get container status \"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a\": rpc error: code = NotFound desc = could not find container \"4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a\": container with ID starting with 4bc04e343c2b27c113758c30f41400de457f6133158654bc561edd130fe1847a not found: ID does not exist" Jun 11 16:50:32.994296 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994256 2557 scope.go:117] "RemoveContainer" containerID="8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885" Jun 11 16:50:32.994458 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994429 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885"} err="failed to get container status \"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885\": rpc error: code = NotFound desc = could not find container \"8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885\": container with ID starting with 8a8e7099f8ddd34a19fc215e04dbd95419a0f620b318f600b568920d321cb885 not found: ID does not exist" Jun 11 16:50:32.994504 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994459 2557 scope.go:117] "RemoveContainer" containerID="6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5" Jun 11 16:50:32.994651 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994634 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5"} err="failed to get container status \"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5\": rpc error: code = NotFound desc = could not find container \"6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5\": container with ID starting with 6f123b28ef36a5d727d124482771a718a0351b41c71a7c2f417bbe98271aa3b5 not found: ID does not exist" Jun 11 16:50:32.994697 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994651 2557 scope.go:117] "RemoveContainer" containerID="a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6" Jun 11 16:50:32.994850 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994821 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6"} err="failed to get container status \"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6\": rpc error: code = NotFound desc = could not find container \"a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6\": container with ID starting with a94de1af7bdfec1056a4fd17b40189995051b4423ee2121fd0ed1de403bfe9a6 not found: ID does not exist" Jun 11 16:50:32.994850 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.994842 2557 scope.go:117] "RemoveContainer" containerID="c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de" Jun 11 16:50:32.995051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.995036 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de"} err="failed to get container status \"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de\": rpc error: code = NotFound desc = could not find container \"c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de\": container with ID starting with c237ec61683dd2a486988c3d0696a46e6c375fe8f4a7647e88a0d4b4a66238de not found: ID does not exist" Jun 11 16:50:32.995096 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.995051 2557 scope.go:117] "RemoveContainer" containerID="84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5" Jun 11 16:50:32.995270 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:32.995251 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5"} err="failed to get container status \"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5\": rpc error: code = NotFound desc = could not find container \"84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5\": container with ID starting with 84c6692511eebfd13a5b17b4185cfc89f6646749d2b96b37413c7867449295e5 not found: ID does not exist" Jun 11 16:50:33.002282 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002260 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:33.002562 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002550 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-web" Jun 11 16:50:33.002620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002564 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-web" Jun 11 16:50:33.002620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002574 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e069e02-ad4b-4482-abd1-992cbb3af597" containerName="console" Jun 11 16:50:33.002620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002597 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e069e02-ad4b-4482-abd1-992cbb3af597" containerName="console" Jun 11 16:50:33.002620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002611 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="alertmanager" Jun 11 16:50:33.002620 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002617 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="alertmanager" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002624 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="config-reloader" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002628 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="config-reloader" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002636 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" containerName="registry" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002642 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" containerName="registry" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002649 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="prom-label-proxy" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002654 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="prom-label-proxy" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002664 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="init-config-reloader" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002673 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="init-config-reloader" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002682 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002689 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002699 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-metric" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002704 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-metric" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002751 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e069e02-ad4b-4482-abd1-992cbb3af597" containerName="console" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002757 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-metric" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002763 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="prom-label-proxy" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002768 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9de10675-cbff-4637-93ef-a69149df0c7b" containerName="registry" Jun 11 16:50:33.002770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002773 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy-web" Jun 11 16:50:33.003226 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002778 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="kube-rbac-proxy" Jun 11 16:50:33.003226 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002785 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="alertmanager" Jun 11 16:50:33.003226 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.002790 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" containerName="config-reloader" Jun 11 16:50:33.006492 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.006469 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.008787 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.008766 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 16:50:33.008787 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.008781 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 16:50:33.008927 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.008773 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 16:50:33.009019 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009002 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 16:50:33.009188 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009174 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 16:50:33.009223 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009185 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 16:50:33.009260 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009244 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-v99mc\"" Jun 11 16:50:33.009296 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009190 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 16:50:33.009564 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.009535 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 16:50:33.014075 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.014056 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 16:50:33.027970 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.027950 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:33.083565 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083571 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-web-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjg5r\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-kube-api-access-jjg5r\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083729 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-out\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.083939 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.084090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.084090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.083985 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.084090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.084008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-volume\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184411 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184485 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184512 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184509 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184548 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184620 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184740 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-volume\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184826 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.184879 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-web-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.185220 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184887 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjg5r\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-kube-api-access-jjg5r\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.185220 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.184938 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-out\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.185689 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.185414 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.185828 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.185786 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.186142 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.186050 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.187704 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.187550 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-out\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.187864 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.187834 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.187964 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.187953 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.188022 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.187998 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.188075 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.188001 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.188268 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.188251 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.188444 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.188427 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.188614 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.188566 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-config-volume\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.189850 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.189832 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/20b03e5d-4c8a-4db1-aa34-044945716ac5-web-config\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.199822 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.199796 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjg5r\" (UniqueName: \"kubernetes.io/projected/20b03e5d-4c8a-4db1-aa34-044945716ac5-kube-api-access-jjg5r\") pod \"alertmanager-main-0\" (UID: \"20b03e5d-4c8a-4db1-aa34-044945716ac5\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.316658 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.316625 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:50:33.460222 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.460194 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:50:33.461678 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:50:33.461650 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b03e5d_4c8a_4db1_aa34_044945716ac5.slice/crio-a3942044eea8715d65e83d685a8837d92cc996be6445f059fec9ce14fdc5c783 WatchSource:0}: Error finding container a3942044eea8715d65e83d685a8837d92cc996be6445f059fec9ce14fdc5c783: Status 404 returned error can't find the container with id a3942044eea8715d65e83d685a8837d92cc996be6445f059fec9ce14fdc5c783 Jun 11 16:50:33.945912 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.945879 2557 generic.go:356] "Generic (PLEG): container finished" podID="20b03e5d-4c8a-4db1-aa34-044945716ac5" containerID="8eb200e35c0cd76195d51a877e6735b90de8d505720c147337ae691fa5b75a9b" exitCode=0 Jun 11 16:50:33.946309 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.945976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerDied","Data":"8eb200e35c0cd76195d51a877e6735b90de8d505720c147337ae691fa5b75a9b"} Jun 11 16:50:33.946309 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:33.946049 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"a3942044eea8715d65e83d685a8837d92cc996be6445f059fec9ce14fdc5c783"} Jun 11 16:50:34.517163 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.517129 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d67312-99e6-4bb1-ab05-8d96e026199a" path="/var/lib/kubelet/pods/05d67312-99e6-4bb1-ab05-8d96e026199a/volumes" Jun 11 16:50:34.956240 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956127 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"6f0fe6ae921de4d7ef3d9e7cf8edcc1f2f865b8fd2781411bef72aaa30f45dda"} Jun 11 16:50:34.956719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956241 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"a038f05f9a8d37d0f3e896597c90072d3ac8b5382c4824ca38bda9308ddc2ab1"} Jun 11 16:50:34.956719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956281 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"4f53276673761c40234ee5b3d3b2b333557f6483141dc9804025e8e1080c79d4"} Jun 11 16:50:34.956719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"e488538a7bd93c175d3cc8ced0b3ee59b9c35f638903ca5ee634b4749f48ce3f"} Jun 11 16:50:34.956719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956450 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"8d148252e8285662262d31cfc976d0cf754d74d12004ec7c885420f9e323ea4e"} Jun 11 16:50:34.956719 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.956504 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"20b03e5d-4c8a-4db1-aa34-044945716ac5","Type":"ContainerStarted","Data":"b88d672f3c38cc95937c9f0d796e6a506dcce9b2cb27d99397ebd40451cc4859"} Jun 11 16:50:34.985347 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:34.985134 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9695179449999998 podStartE2EDuration="2.985110946s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.593001ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:50:34.983659826 +0000 UTC m=+390.872791203" watchObservedRunningTime="2026-06-11 16:50:34.985110946 +0000 UTC m=+390.874242325" Jun 11 16:50:35.083472 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.083431 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.083837 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="prometheus" containerID="cri-o://0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" gracePeriod=600 Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.084044 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="thanos-sidecar" containerID="cri-o://521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" gracePeriod=600 Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.084067 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-web" containerID="cri-o://78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" gracePeriod=600 Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.084145 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="config-reloader" containerID="cri-o://c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" gracePeriod=600 Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.084199 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" gracePeriod=600 Jun 11 16:50:35.084378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.084269 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy" containerID="cri-o://9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" gracePeriod=600 Jun 11 16:50:35.335800 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.335771 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.407942 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.408358 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.408776 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.408830 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thmrz\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409154 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409203 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409233 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409276 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409310 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409344 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409483 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409550 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409629 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409666 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409702 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409728 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409751 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409811 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409839 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.409950 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.409876 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy\") pod \"f7951afc-0bb4-4306-9268-9c7f278e6532\" (UID: \"f7951afc-0bb4-4306-9268-9c7f278e6532\") " Jun 11 16:50:35.410491 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.410137 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.410491 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.410157 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:50:35.410978 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.410728 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.411159 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.411135 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.413768 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.413465 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:50:35.413768 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.413746 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.414003 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.413821 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.414064 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414036 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.414123 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414056 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.414179 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414150 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config" (OuterVolumeSpecName: "config") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.414317 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414289 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz" (OuterVolumeSpecName: "kube-api-access-thmrz") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "kube-api-access-thmrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:50:35.414383 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414342 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.414443 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414386 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.414725 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414701 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.414927 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.414899 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.415432 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.415411 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:50:35.416386 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.416366 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out" (OuterVolumeSpecName: "config-out") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:50:35.416504 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.416483 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.416562 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.416545 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.426752 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.426727 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config" (OuterVolumeSpecName: "web-config") pod "f7951afc-0bb4-4306-9268-9c7f278e6532" (UID: "f7951afc-0bb4-4306-9268-9c7f278e6532"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:50:35.511083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511053 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-tls-assets\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511078 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-metrics-client-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511091 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511103 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511112 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511121 2557 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-metrics-client-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511130 2557 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511137 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511147 2557 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511157 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-web-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511165 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511173 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511181 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-config-out\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511192 2557 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-kube-rbac-proxy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511199 2557 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-grpc-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511207 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thmrz\" (UniqueName: \"kubernetes.io/projected/f7951afc-0bb4-4306-9268-9c7f278e6532-kube-api-access-thmrz\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511226 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-db\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511234 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f7951afc-0bb4-4306-9268-9c7f278e6532-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.511291 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.511242 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f7951afc-0bb4-4306-9268-9c7f278e6532-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:50:35.963113 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963078 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" exitCode=0 Jun 11 16:50:35.963113 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963108 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" exitCode=0 Jun 11 16:50:35.963113 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963117 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" exitCode=0 Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963124 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" exitCode=0 Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963129 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" exitCode=0 Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963137 2557 generic.go:356] "Generic (PLEG): container finished" podID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" exitCode=0 Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963151 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963253 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963282 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963329 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963343 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963376 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} Jun 11 16:50:35.963567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.963415 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f7951afc-0bb4-4306-9268-9c7f278e6532","Type":"ContainerDied","Data":"fdce7a2234bbf2d88e0a537e8bded6693501f77e34906b588174462d9ae00be5"} Jun 11 16:50:35.971303 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.971285 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:35.978229 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.978213 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:35.985063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.985046 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:35.992558 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.992538 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:35.999811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:35.999790 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.007278 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.007245 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.014398 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014298 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:50:36.014694 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014664 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.014806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014745 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="config-reloader" Jun 11 16:50:36.014806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014762 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="config-reloader" Jun 11 16:50:36.014806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014780 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-web" Jun 11 16:50:36.014806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014789 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-web" Jun 11 16:50:36.014806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014805 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="thanos-sidecar" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014814 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="thanos-sidecar" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014831 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="prometheus" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014841 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="prometheus" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014849 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-thanos" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014857 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-thanos" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014867 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014875 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014894 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="init-config-reloader" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014902 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="init-config-reloader" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.014940 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014964 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014981 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-web" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015003 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy-thanos" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015014 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="prometheus" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015026 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="config-reloader" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015034 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="kube-rbac-proxy" Jun 11 16:50:36.015051 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015044 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" containerName="thanos-sidecar" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.014982 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.015311 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015330 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015345 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.015556 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015573 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.015811 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015607 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.016090 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.015848 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.016090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015864 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.016090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.015878 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.016182 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.016115 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.016182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016136 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.016182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016150 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.016386 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.016370 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.016418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016389 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.016418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016402 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.016618 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:50:36.016597 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.016761 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016620 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.016761 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016633 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.016852 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016836 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.016885 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.016852 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.017036 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017021 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.017077 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017037 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.017206 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017192 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.017240 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017206 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.017378 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017356 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.017418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017378 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.017616 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017595 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.017700 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017618 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.017840 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017826 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.017882 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.017840 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.018026 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018009 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.018026 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018025 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.018201 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018186 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.018235 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018202 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.018398 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018383 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.018438 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018398 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.018570 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018552 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.018753 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018572 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.018817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018765 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.018817 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018788 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.019011 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.018992 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.019067 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019013 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.019200 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019180 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.019274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019217 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.019274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019233 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.019625 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019594 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.019720 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019626 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.019878 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019859 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.019878 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.019877 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.020104 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020085 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.020151 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020106 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.020331 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020310 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.020370 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020332 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.020567 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020550 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.020619 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020569 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.020805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020786 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.020805 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.020803 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.021041 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021019 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.021041 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021041 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.021275 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021256 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.021337 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021275 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.021476 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021460 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.021541 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021476 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.021725 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021706 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.021725 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021724 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.021928 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021911 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.021928 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.021926 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.022169 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022147 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.022213 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022171 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.022410 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022391 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.022457 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022412 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.022733 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022712 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.022733 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022732 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.022966 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022947 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.023026 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.022968 2557 scope.go:117] "RemoveContainer" containerID="9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae" Jun 11 16:50:36.023191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023173 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae"} err="failed to get container status \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": rpc error: code = NotFound desc = could not find container \"9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae\": container with ID starting with 9572b4162a5af2646db5b04a7e4c594b7eb15de8cbb33b2819d3251233f74bae not found: ID does not exist" Jun 11 16:50:36.023239 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023192 2557 scope.go:117] "RemoveContainer" containerID="9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a" Jun 11 16:50:36.023382 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023366 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a"} err="failed to get container status \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": rpc error: code = NotFound desc = could not find container \"9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a\": container with ID starting with 9f23a1320706b534d26cb3336c5e2c39bc3e762810404a168d2b6ac91dda868a not found: ID does not exist" Jun 11 16:50:36.023420 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023382 2557 scope.go:117] "RemoveContainer" containerID="78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a" Jun 11 16:50:36.023629 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023608 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a"} err="failed to get container status \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": rpc error: code = NotFound desc = could not find container \"78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a\": container with ID starting with 78cd39b3e2bcd803e7db31a372da8b6b8893a778808540b8c83f721368c5d00a not found: ID does not exist" Jun 11 16:50:36.023702 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023631 2557 scope.go:117] "RemoveContainer" containerID="521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802" Jun 11 16:50:36.023765 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023734 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 11 16:50:36.023815 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023768 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 11 16:50:36.023815 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023777 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 11 16:50:36.023916 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023838 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 11 16:50:36.023916 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023777 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 11 16:50:36.023916 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.023877 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fvk45\"" Jun 11 16:50:36.024167 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024140 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802"} err="failed to get container status \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": rpc error: code = NotFound desc = could not find container \"521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802\": container with ID starting with 521063be1759c118026135246922f713c9ecc70d3517fecb87e40757560bf802 not found: ID does not exist" Jun 11 16:50:36.024167 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024167 2557 scope.go:117] "RemoveContainer" containerID="c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd" Jun 11 16:50:36.024303 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024184 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 16:50:36.024407 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024379 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd"} err="failed to get container status \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": rpc error: code = NotFound desc = could not find container \"c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd\": container with ID starting with c67e8132274e5ef561e7c5504a728d7c69ea9ceef8edb307cf9a9049c0aa60cd not found: ID does not exist" Jun 11 16:50:36.024505 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024408 2557 scope.go:117] "RemoveContainer" containerID="0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45" Jun 11 16:50:36.024767 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024743 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45"} err="failed to get container status \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": rpc error: code = NotFound desc = could not find container \"0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45\": container with ID starting with 0f9697d2e97e87a4d2803785f8ecd77ed1111546994e1372b9daf8eed28aae45 not found: ID does not exist" Jun 11 16:50:36.024839 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.024768 2557 scope.go:117] "RemoveContainer" containerID="1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403" Jun 11 16:50:36.025071 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.025049 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403"} err="failed to get container status \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": rpc error: code = NotFound desc = could not find container \"1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403\": container with ID starting with 1d788c49d197fdda6c8e756572c90c6c3b05657583512519d56199c3b57a0403 not found: ID does not exist" Jun 11 16:50:36.026485 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.026450 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 16:50:36.034449 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.034428 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 11 16:50:36.038888 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.038857 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:36.040142 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.040123 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:50:36.048941 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.048919 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:36.091823 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.091792 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:36.094735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.094716 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.097089 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097070 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 16:50:36.097089 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097078 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 16:50:36.097228 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097144 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 16:50:36.097397 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097383 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 16:50:36.097480 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097435 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 16:50:36.097691 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.097679 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 16:50:36.099610 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.099573 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 16:50:36.099674 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.099616 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 16:50:36.100617 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.100596 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 16:50:36.100788 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.100768 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 16:50:36.100788 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.100781 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 16:50:36.101329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.101315 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sp6sw\"" Jun 11 16:50:36.101381 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.101328 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3is6ea7unc174\"" Jun 11 16:50:36.103351 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.103336 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 16:50:36.105353 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.105336 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 16:50:36.108642 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.108627 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 16:50:36.114667 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114648 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114756 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114756 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114714 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114858 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114858 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114793 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114858 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114838 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnhqx\" (UniqueName: \"kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.114994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.114920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.121743 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.121721 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:36.215810 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.215960 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215816 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.215960 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.215960 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.215960 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.215960 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-web-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.215979 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnhqx\" (UniqueName: \"kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.216143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216035 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216068 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216110 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216210 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216236 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216329 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216400 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216471 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-config-out\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.216660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.217102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216700 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98s6m\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-kube-api-access-98s6m\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.217102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.216929 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.217102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.217055 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.217201 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.217139 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.218413 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.218397 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.218789 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.218775 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.225442 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.225411 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnhqx\" (UniqueName: \"kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx\") pod \"console-5569db675c-p8vjn\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.317327 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317327 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317323 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-web-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317399 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317453 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317477 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317498 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317535 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.317572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317605 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317757 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-config-out\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317771 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317785 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317811 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318033 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.317861 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98s6m\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-kube-api-access-98s6m\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318511 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.318342 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.318883 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.318857 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.319470 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.319441 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.320533 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.320433 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.320979 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.321060 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-web-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.321243 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.321341 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.321396 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.321573 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-config\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.323771 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/002d4b8b-0c30-4de2-bf55-dd0091eac874-config-out\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.323972 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.323963 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.324414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.324038 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.324414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.324081 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.324414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.324190 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.327897 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.324554 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.327897 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.325008 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/002d4b8b-0c30-4de2-bf55-dd0091eac874-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.327897 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.326971 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98s6m\" (UniqueName: \"kubernetes.io/projected/002d4b8b-0c30-4de2-bf55-dd0091eac874-kube-api-access-98s6m\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.329870 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.329822 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:36.330115 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.330096 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/002d4b8b-0c30-4de2-bf55-dd0091eac874-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"002d4b8b-0c30-4de2-bf55-dd0091eac874\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.408981 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.407915 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:36.474359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.474327 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:50:36.476003 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:50:36.475974 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb29810_adda_428b_b00b_8f3af16346ae.slice/crio-fe3ae1277c9100b6bd430355ebace2f797c7b6220272da2189051b7db89775d1 WatchSource:0}: Error finding container fe3ae1277c9100b6bd430355ebace2f797c7b6220272da2189051b7db89775d1: Status 404 returned error can't find the container with id fe3ae1277c9100b6bd430355ebace2f797c7b6220272da2189051b7db89775d1 Jun 11 16:50:36.519219 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.518768 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7951afc-0bb4-4306-9268-9c7f278e6532" path="/var/lib/kubelet/pods/f7951afc-0bb4-4306-9268-9c7f278e6532/volumes" Jun 11 16:50:36.557035 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.557013 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:50:36.559016 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:50:36.558991 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod002d4b8b_0c30_4de2_bf55_dd0091eac874.slice/crio-a2eea71f6f9c473b04bf35fb1b9fe0db699cf712c5abee831100eb3006220cad WatchSource:0}: Error finding container a2eea71f6f9c473b04bf35fb1b9fe0db699cf712c5abee831100eb3006220cad: Status 404 returned error can't find the container with id a2eea71f6f9c473b04bf35fb1b9fe0db699cf712c5abee831100eb3006220cad Jun 11 16:50:36.969262 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.969227 2557 generic.go:356] "Generic (PLEG): container finished" podID="002d4b8b-0c30-4de2-bf55-dd0091eac874" containerID="d5b9b7c6936013bdc3523065541da9556c5a6cc37a773610828b3012938f84d7" exitCode=0 Jun 11 16:50:36.969741 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.969321 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerDied","Data":"d5b9b7c6936013bdc3523065541da9556c5a6cc37a773610828b3012938f84d7"} Jun 11 16:50:36.969741 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.969392 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"a2eea71f6f9c473b04bf35fb1b9fe0db699cf712c5abee831100eb3006220cad"} Jun 11 16:50:36.971855 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.971831 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5569db675c-p8vjn" event={"ID":"7fb29810-adda-428b-b00b-8f3af16346ae","Type":"ContainerStarted","Data":"a80bcb01355fb0bfceda8fd3d99ebf380f5076e64b6f1109ed630e6d9b0b3839"} Jun 11 16:50:36.971957 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:36.971871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5569db675c-p8vjn" event={"ID":"7fb29810-adda-428b-b00b-8f3af16346ae","Type":"ContainerStarted","Data":"fe3ae1277c9100b6bd430355ebace2f797c7b6220272da2189051b7db89775d1"} Jun 11 16:50:37.013083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.013037 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5569db675c-p8vjn" podStartSLOduration=2.013021417 podStartE2EDuration="2.013021417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:50:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:50:37.012626388 +0000 UTC m=+392.901757768" watchObservedRunningTime="2026-06-11 16:50:37.013021417 +0000 UTC m=+392.902152868" Jun 11 16:50:37.980643 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980605 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"97609e9ce01512e66bcc352fbaf77566495bb44bc88b284ba8d5cbcd518a66e1"} Jun 11 16:50:37.981030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980671 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"91921015a197d8bb37f834f39a5c9271a4db17bd6f7a0d7d4dd9b168115b000c"} Jun 11 16:50:37.981030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980699 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"37753823f2c81b0a8a076dc0862185bd322ec8ddefe42ca370fff945d0eb14b5"} Jun 11 16:50:37.981030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980724 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"31183e9f513ed77dbbd1d520d326db47a6c14a98fb9c9ebb5e2137a324de0dff"} Jun 11 16:50:37.981030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"2131fb8dfa74d9f06f48c7c4a6ca52af9d42b627bc11d9071f8fb764222b5e01"} Jun 11 16:50:37.981030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:37.980773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"002d4b8b-0c30-4de2-bf55-dd0091eac874","Type":"ContainerStarted","Data":"ef33a425db98f700ecb90746dbd5420c0532348fae3bfc90b9f60c9dfb3cee91"} Jun 11 16:50:38.011901 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:38.011851 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.9929420690000001 podStartE2EDuration="2.011832043s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.889974ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:50:38.010192972 +0000 UTC m=+393.899324362" watchObservedRunningTime="2026-06-11 16:50:38.011832043 +0000 UTC m=+393.900963427" Jun 11 16:50:41.408623 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:41.408573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:50:46.330882 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:46.330841 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:46.331246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:46.330909 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:46.335851 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:46.335828 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:50:47.024415 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:50:47.024384 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:51:36.408868 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:51:36.408772 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:51:36.424946 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:51:36.424916 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:51:37.243866 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:51:37.243840 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:52:50.270031 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.267090 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv"] Jun 11 16:52:50.272027 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.272001 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.286846 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.286819 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 16:52:50.286957 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.286924 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 16:52:50.286957 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.286946 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 16:52:50.287035 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.286965 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 16:52:50.287094 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.287075 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vh62b\"" Jun 11 16:52:50.298078 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.298052 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv"] Jun 11 16:52:50.377455 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.377417 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-manifests\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.377661 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.377462 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vfmq\" (UniqueName: \"kubernetes.io/projected/404ac9b8-25ce-41b4-9700-196fe5af3aca-kube-api-access-6vfmq\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.377661 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.377555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-webhook-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.377661 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.377613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.377772 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.377722 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-tmp\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.478728 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.478695 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vfmq\" (UniqueName: \"kubernetes.io/projected/404ac9b8-25ce-41b4-9700-196fe5af3aca-kube-api-access-6vfmq\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.478915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.478755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-webhook-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.478915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.478790 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.478915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.478862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-tmp\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.478915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.478898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-manifests\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.479334 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.479308 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-manifests\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.481149 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.481121 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404ac9b8-25ce-41b4-9700-196fe5af3aca-tmp\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.481412 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.481392 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-webhook-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.481488 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.481394 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404ac9b8-25ce-41b4-9700-196fe5af3aca-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.488872 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.488848 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vfmq\" (UniqueName: \"kubernetes.io/projected/404ac9b8-25ce-41b4-9700-196fe5af3aca-kube-api-access-6vfmq\") pod \"opendatahub-operator-controller-manager-f54898884-z6trv\" (UID: \"404ac9b8-25ce-41b4-9700-196fe5af3aca\") " pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.582185 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.582102 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:50.718447 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:50.718422 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv"] Jun 11 16:52:50.720652 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:52:50.720626 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404ac9b8_25ce_41b4_9700_196fe5af3aca.slice/crio-6f83e231cf1f12cf865fa485b73a266f3fae992831e8a5878d48398ef710b54a WatchSource:0}: Error finding container 6f83e231cf1f12cf865fa485b73a266f3fae992831e8a5878d48398ef710b54a: Status 404 returned error can't find the container with id 6f83e231cf1f12cf865fa485b73a266f3fae992831e8a5878d48398ef710b54a Jun 11 16:52:51.535094 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:51.535005 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" event={"ID":"404ac9b8-25ce-41b4-9700-196fe5af3aca","Type":"ContainerStarted","Data":"6f83e231cf1f12cf865fa485b73a266f3fae992831e8a5878d48398ef710b54a"} Jun 11 16:52:54.551723 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:54.551684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" event={"ID":"404ac9b8-25ce-41b4-9700-196fe5af3aca","Type":"ContainerStarted","Data":"5a74439727d90584c37feb36e9c696be03891037bd2fe93e087f557c40fd62a8"} Jun 11 16:52:55.557814 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:55.557774 2557 generic.go:356] "Generic (PLEG): container finished" podID="404ac9b8-25ce-41b4-9700-196fe5af3aca" containerID="5a74439727d90584c37feb36e9c696be03891037bd2fe93e087f557c40fd62a8" exitCode=0 Jun 11 16:52:55.558288 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:55.557820 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" event={"ID":"404ac9b8-25ce-41b4-9700-196fe5af3aca","Type":"ContainerDied","Data":"5a74439727d90584c37feb36e9c696be03891037bd2fe93e087f557c40fd62a8"} Jun 11 16:52:56.563799 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:56.563763 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" event={"ID":"404ac9b8-25ce-41b4-9700-196fe5af3aca","Type":"ContainerStarted","Data":"4bae817736e7a388ac3c0aaf410a4961565dcf6993fec6c35fe3e0af39c6b1ce"} Jun 11 16:52:56.564272 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:56.563902 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:52:56.588735 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:52:56.588675 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" podStartSLOduration=2.455874663 podStartE2EDuration="6.588657526s" totalImagesPullingTime="3.11897954s" totalInitContainerRuntime="1.013803323s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:52:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:52:56.587082117 +0000 UTC m=+532.476213496" watchObservedRunningTime="2026-06-11 16:52:56.588657526 +0000 UTC m=+532.477788902" Jun 11 16:53:07.570807 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:07.570775 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f54898884-z6trv" Jun 11 16:53:10.803408 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.803368 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj"] Jun 11 16:53:10.815866 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.815840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.829449 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.829429 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:53:10.830905 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.830887 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 11 16:53:10.831771 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.831757 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 11 16:53:10.831836 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.831786 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 11 16:53:10.831836 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.831786 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 11 16:53:10.831945 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.831839 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6g97g\"" Jun 11 16:53:10.857288 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.857263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj"] Jun 11 16:53:10.862289 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.862265 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.862289 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.862290 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-metrics-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.862472 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.862330 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-manager-config\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.862472 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.862385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m4t9\" (UniqueName: \"kubernetes.io/projected/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-kube-api-access-6m4t9\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.963475 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.963436 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.963475 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.963475 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-metrics-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.963770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.963527 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-manager-config\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.963770 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.963546 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m4t9\" (UniqueName: \"kubernetes.io/projected/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-kube-api-access-6m4t9\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.964247 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.964228 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-manager-config\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.966050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.966025 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-metrics-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.966149 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.966052 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-cert\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:10.978299 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:10.978277 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m4t9\" (UniqueName: \"kubernetes.io/projected/a85ac72d-d3e6-4de9-a53f-5f1c58f10786-kube-api-access-6m4t9\") pod \"lws-controller-manager-57b9974d87-4p7sj\" (UID: \"a85ac72d-d3e6-4de9-a53f-5f1c58f10786\") " pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:11.125274 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:11.125196 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:11.280775 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:11.280745 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj"] Jun 11 16:53:11.283049 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:53:11.283021 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85ac72d_d3e6_4de9_a53f_5f1c58f10786.slice/crio-ccd2cbf67e1be8b51d135245182966700a8e269de46bdd2733190e77401e2910 WatchSource:0}: Error finding container ccd2cbf67e1be8b51d135245182966700a8e269de46bdd2733190e77401e2910: Status 404 returned error can't find the container with id ccd2cbf67e1be8b51d135245182966700a8e269de46bdd2733190e77401e2910 Jun 11 16:53:11.628970 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:11.628935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" event={"ID":"a85ac72d-d3e6-4de9-a53f-5f1c58f10786","Type":"ContainerStarted","Data":"ccd2cbf67e1be8b51d135245182966700a8e269de46bdd2733190e77401e2910"} Jun 11 16:53:14.646142 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:14.646101 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" event={"ID":"a85ac72d-d3e6-4de9-a53f-5f1c58f10786","Type":"ContainerStarted","Data":"fbb9a369868046a4b31c54f12d64128b9445704f1083e8b5b66ed2485d891fa1"} Jun 11 16:53:14.646525 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:14.646184 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:53:14.667478 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:14.667430 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" podStartSLOduration=1.95016036 podStartE2EDuration="4.667415986s" totalImagesPullingTime="2.717255626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:53:14.666826349 +0000 UTC m=+550.555957725" watchObservedRunningTime="2026-06-11 16:53:14.667415986 +0000 UTC m=+550.556547363" Jun 11 16:53:16.441746 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.441714 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw"] Jun 11 16:53:16.444418 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.444400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.449140 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.449118 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 16:53:16.449417 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.449401 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b7x4l\"" Jun 11 16:53:16.451613 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.451592 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 16:53:16.469804 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.469777 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw"] Jun 11 16:53:16.511306 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.511270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tmp\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.511476 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.511322 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tls-certs\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.511476 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.511408 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6bqc\" (UniqueName: \"kubernetes.io/projected/8fa49174-5b8a-4fa9-ace4-71a8feefd759-kube-api-access-t6bqc\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.612884 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.612841 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tmp\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.612884 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.612889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tls-certs\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.613106 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.612946 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6bqc\" (UniqueName: \"kubernetes.io/projected/8fa49174-5b8a-4fa9-ace4-71a8feefd759-kube-api-access-t6bqc\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.615365 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.615330 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tmp\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.615502 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.615489 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fa49174-5b8a-4fa9-ace4-71a8feefd759-tls-certs\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.622280 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.622254 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6bqc\" (UniqueName: \"kubernetes.io/projected/8fa49174-5b8a-4fa9-ace4-71a8feefd759-kube-api-access-t6bqc\") pod \"kube-auth-proxy-588969d7f4-5vlxw\" (UID: \"8fa49174-5b8a-4fa9-ace4-71a8feefd759\") " pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.753932 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.753901 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" Jun 11 16:53:16.885726 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:16.885522 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw"] Jun 11 16:53:16.888434 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:53:16.888404 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa49174_5b8a_4fa9_ace4_71a8feefd759.slice/crio-43617cdcaaf71bfa6fabf864a6b9b07ed5cbe88b1601b4ef4f918fef858d1698 WatchSource:0}: Error finding container 43617cdcaaf71bfa6fabf864a6b9b07ed5cbe88b1601b4ef4f918fef858d1698: Status 404 returned error can't find the container with id 43617cdcaaf71bfa6fabf864a6b9b07ed5cbe88b1601b4ef4f918fef858d1698 Jun 11 16:53:17.665215 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:17.665172 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" event={"ID":"8fa49174-5b8a-4fa9-ace4-71a8feefd759","Type":"ContainerStarted","Data":"43617cdcaaf71bfa6fabf864a6b9b07ed5cbe88b1601b4ef4f918fef858d1698"} Jun 11 16:53:19.676907 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:19.676863 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" event={"ID":"8fa49174-5b8a-4fa9-ace4-71a8feefd759","Type":"ContainerStarted","Data":"836107ea45a485ec58b482d6b54a4d60fc0f8d8a077cdc8639dddc4ed09b8fdd"} Jun 11 16:53:19.696494 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:19.696442 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-588969d7f4-5vlxw" podStartSLOduration=1.013173869 podStartE2EDuration="3.696425755s" totalImagesPullingTime="2.683251886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:53:19.693000987 +0000 UTC m=+555.582132365" watchObservedRunningTime="2026-06-11 16:53:19.696425755 +0000 UTC m=+555.585557131" Jun 11 16:53:25.652234 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:53:25.652202 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57b9974d87-4p7sj" Jun 11 16:54:04.574449 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:04.574339 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:54:04.579087 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:04.576913 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:54:38.994494 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:38.994401 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:54:53.371474 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.371436 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn"] Jun 11 16:54:53.374794 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.374777 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.377299 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.377280 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 16:54:53.377299 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.377293 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 16:54:53.377467 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.377421 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l78nt\"" Jun 11 16:54:53.387947 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.387924 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn"] Jun 11 16:54:53.401785 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.401753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.401906 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.401806 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96v7l\" (UniqueName: \"kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.502819 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.502779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96v7l\" (UniqueName: \"kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.503014 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.502897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.503348 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.503326 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.519108 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.519083 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96v7l\" (UniqueName: \"kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.684402 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.684312 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:54:53.817545 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.817519 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn"] Jun 11 16:54:53.820399 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:54:53.820373 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9cd74e_297a_4671_aeca_8affa476c1c0.slice/crio-9cdecace04181303bc643dfd0aabc47d8f1a3d490caea41a5a3947bd9b99b3c9 WatchSource:0}: Error finding container 9cdecace04181303bc643dfd0aabc47d8f1a3d490caea41a5a3947bd9b99b3c9: Status 404 returned error can't find the container with id 9cdecace04181303bc643dfd0aabc47d8f1a3d490caea41a5a3947bd9b99b3c9 Jun 11 16:54:53.822802 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:53.822786 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:54:54.098361 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:54.098327 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" event={"ID":"3e9cd74e-297a-4671-aeca-8affa476c1c0","Type":"ContainerStarted","Data":"9cdecace04181303bc643dfd0aabc47d8f1a3d490caea41a5a3947bd9b99b3c9"} Jun 11 16:54:59.124969 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:59.124936 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" event={"ID":"3e9cd74e-297a-4671-aeca-8affa476c1c0","Type":"ContainerStarted","Data":"11bf0dd948446a2d1c28560e058d886e01161e6c5a1028ef99d22dbe26bc5936"} Jun 11 16:54:59.125361 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:54:59.125072 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:01.151937 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.151882 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" podStartSLOduration=3.813667431 podStartE2EDuration="8.151862288s" totalImagesPullingTime="4.338194857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:54:59.151294454 +0000 UTC m=+655.040425843" watchObservedRunningTime="2026-06-11 16:55:01.151862288 +0000 UTC m=+657.040993664" Jun 11 16:55:01.152414 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.152028 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4"] Jun 11 16:55:01.155843 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.155821 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.160183 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.160163 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qfjql\"" Jun 11 16:55:01.161683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.161645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 11 16:55:01.161944 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.161921 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 11 16:55:01.163915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.163893 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4"] Jun 11 16:55:01.264113 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.264063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.264113 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.264125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.264331 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.264169 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gzrz\" (UniqueName: \"kubernetes.io/projected/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-kube-api-access-7gzrz\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.365411 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.365370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.365613 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.365445 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.365613 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.365469 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gzrz\" (UniqueName: \"kubernetes.io/projected/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-kube-api-access-7gzrz\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.365969 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.365949 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.367883 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.367864 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.381206 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.381178 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gzrz\" (UniqueName: \"kubernetes.io/projected/cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd-kube-api-access-7gzrz\") pod \"kuadrant-console-plugin-66b5774985-jshq4\" (UID: \"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.476785 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.476753 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" Jun 11 16:55:01.635827 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:01.635796 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4"] Jun 11 16:55:01.638625 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:55:01.638598 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbcb9ec6_8798_4004_bd68_49f8c1dbe0fd.slice/crio-cd8efba6cacd59642cafa74d24a33de9dddc8874f6bd69cbc400b82ac6960ca0 WatchSource:0}: Error finding container cd8efba6cacd59642cafa74d24a33de9dddc8874f6bd69cbc400b82ac6960ca0: Status 404 returned error can't find the container with id cd8efba6cacd59642cafa74d24a33de9dddc8874f6bd69cbc400b82ac6960ca0 Jun 11 16:55:02.142444 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:02.142409 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" event={"ID":"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd","Type":"ContainerStarted","Data":"cd8efba6cacd59642cafa74d24a33de9dddc8874f6bd69cbc400b82ac6960ca0"} Jun 11 16:55:04.016083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.016040 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5569db675c-p8vjn" podUID="7fb29810-adda-428b-b00b-8f3af16346ae" containerName="console" containerID="cri-o://a80bcb01355fb0bfceda8fd3d99ebf380f5076e64b6f1109ed630e6d9b0b3839" gracePeriod=15 Jun 11 16:55:04.158080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.158045 2557 generic.go:356] "Generic (PLEG): container finished" podID="7fb29810-adda-428b-b00b-8f3af16346ae" containerID="a80bcb01355fb0bfceda8fd3d99ebf380f5076e64b6f1109ed630e6d9b0b3839" exitCode=0 Jun 11 16:55:04.158242 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.158118 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5569db675c-p8vjn" event={"ID":"7fb29810-adda-428b-b00b-8f3af16346ae","Type":"ContainerDied","Data":"a80bcb01355fb0bfceda8fd3d99ebf380f5076e64b6f1109ed630e6d9b0b3839"} Jun 11 16:55:04.311360 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.311331 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:55:04.394594 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394477 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnhqx\" (UniqueName: \"kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394594 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394536 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394594 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394572 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394875 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394643 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394875 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394710 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394875 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394769 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.394875 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.394812 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca\") pod \"7fb29810-adda-428b-b00b-8f3af16346ae\" (UID: \"7fb29810-adda-428b-b00b-8f3af16346ae\") " Jun 11 16:55:04.395339 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.395210 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:55:04.395477 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.395388 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config" (OuterVolumeSpecName: "console-config") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:55:04.395477 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.395399 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:55:04.395867 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.395741 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:55:04.397490 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.397456 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx" (OuterVolumeSpecName: "kube-api-access-lnhqx") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "kube-api-access-lnhqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:55:04.398008 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.397951 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:55:04.398355 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.398150 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7fb29810-adda-428b-b00b-8f3af16346ae" (UID: "7fb29810-adda-428b-b00b-8f3af16346ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:55:04.495848 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495809 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-trusted-ca-bundle\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.495848 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495845 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-oauth-serving-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.496063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495860 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-oauth-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.496063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495874 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-console-config\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.496063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495889 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7fb29810-adda-428b-b00b-8f3af16346ae-service-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.496063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495902 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnhqx\" (UniqueName: \"kubernetes.io/projected/7fb29810-adda-428b-b00b-8f3af16346ae-kube-api-access-lnhqx\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:04.496063 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:04.495917 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7fb29810-adda-428b-b00b-8f3af16346ae-console-serving-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:05.166596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:05.166546 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5569db675c-p8vjn" event={"ID":"7fb29810-adda-428b-b00b-8f3af16346ae","Type":"ContainerDied","Data":"fe3ae1277c9100b6bd430355ebace2f797c7b6220272da2189051b7db89775d1"} Jun 11 16:55:05.166596 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:05.166594 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5569db675c-p8vjn" Jun 11 16:55:05.167132 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:05.166641 2557 scope.go:117] "RemoveContainer" containerID="a80bcb01355fb0bfceda8fd3d99ebf380f5076e64b6f1109ed630e6d9b0b3839" Jun 11 16:55:05.201714 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:05.201679 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:55:05.209151 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:05.209111 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5569db675c-p8vjn"] Jun 11 16:55:06.518497 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:06.518463 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fb29810-adda-428b-b00b-8f3af16346ae" path="/var/lib/kubelet/pods/7fb29810-adda-428b-b00b-8f3af16346ae/volumes" Jun 11 16:55:10.132685 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:10.132654 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:12.096198 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.096159 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn"] Jun 11 16:55:12.096798 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.096374 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" containerName="manager" containerID="cri-o://11bf0dd948446a2d1c28560e058d886e01161e6c5a1028ef99d22dbe26bc5936" gracePeriod=2 Jun 11 16:55:12.123316 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.123278 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn"] Jun 11 16:55:12.130250 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:12.130214 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"3e9cd74e-297a-4671-aeca-8affa476c1c0\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-zffdn\": pods \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:12.139290 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139108 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139626 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fb29810-adda-428b-b00b-8f3af16346ae" containerName="console" Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139649 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fb29810-adda-428b-b00b-8f3af16346ae" containerName="console" Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139660 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" containerName="manager" Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139669 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" containerName="manager" Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139789 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fb29810-adda-428b-b00b-8f3af16346ae" containerName="console" Jun 11 16:55:12.140893 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.139800 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" containerName="manager" Jun 11 16:55:12.144307 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.144284 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.146757 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:12.146707 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:12.192118 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.192033 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:12.269944 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.269896 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwznw\" (UniqueName: \"kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.270135 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.270098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.370954 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.370868 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.370954 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.370921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwznw\" (UniqueName: \"kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.371501 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.371321 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.386432 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.386405 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwznw\" (UniqueName: \"kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pq8gw\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.475880 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.475839 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:12.537422 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.537291 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 16:55:12.544168 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.544145 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.565182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.565148 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 16:55:12.673687 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.673605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj749\" (UniqueName: \"kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.673687 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.673673 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.774929 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.774898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj749\" (UniqueName: \"kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.775105 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.774973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.775396 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.775375 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.787520 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.787491 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj749\" (UniqueName: \"kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749\") pod \"kuadrant-operator-controller-manager-86878c769f-57k6t\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:12.856197 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:12.856157 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:24.275899 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.275807 2557 generic.go:356] "Generic (PLEG): container finished" podID="3e9cd74e-297a-4671-aeca-8affa476c1c0" containerID="11bf0dd948446a2d1c28560e058d886e01161e6c5a1028ef99d22dbe26bc5936" exitCode=0 Jun 11 16:55:24.311231 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.311198 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 16:55:24.312853 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:55:24.312825 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a8d8a5_cda8_440c_b039_fa3aab9c9627.slice/crio-5ba20ed6d9d04bf8f02f06ec836eadc3432610f1d04ee2acc1ee3e5859c409fd WatchSource:0}: Error finding container 5ba20ed6d9d04bf8f02f06ec836eadc3432610f1d04ee2acc1ee3e5859c409fd: Status 404 returned error can't find the container with id 5ba20ed6d9d04bf8f02f06ec836eadc3432610f1d04ee2acc1ee3e5859c409fd Jun 11 16:55:24.329944 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.329923 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:24.332666 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:55:24.332595 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b483439_6180_459d_ace3_e615fe9541de.slice/crio-bb91fa86be5ff7e36f169ff82c01abdc39a46a42abd7b7423c65d1d3d5cc3901 WatchSource:0}: Error finding container bb91fa86be5ff7e36f169ff82c01abdc39a46a42abd7b7423c65d1d3d5cc3901: Status 404 returned error can't find the container with id bb91fa86be5ff7e36f169ff82c01abdc39a46a42abd7b7423c65d1d3d5cc3901 Jun 11 16:55:24.379191 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.379165 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:24.381438 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:24.381409 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:24.489872 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.489839 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume\") pod \"3e9cd74e-297a-4671-aeca-8affa476c1c0\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " Jun 11 16:55:24.490034 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.489943 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96v7l\" (UniqueName: \"kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l\") pod \"3e9cd74e-297a-4671-aeca-8affa476c1c0\" (UID: \"3e9cd74e-297a-4671-aeca-8affa476c1c0\") " Jun 11 16:55:24.490195 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.490170 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3e9cd74e-297a-4671-aeca-8affa476c1c0" (UID: "3e9cd74e-297a-4671-aeca-8affa476c1c0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:55:24.492053 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.492033 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l" (OuterVolumeSpecName: "kube-api-access-96v7l") pod "3e9cd74e-297a-4671-aeca-8affa476c1c0" (UID: "3e9cd74e-297a-4671-aeca-8affa476c1c0"). InnerVolumeSpecName "kube-api-access-96v7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:55:24.516430 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:24.516376 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:24.520350 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.520318 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" path="/var/lib/kubelet/pods/3e9cd74e-297a-4671-aeca-8affa476c1c0/volumes" Jun 11 16:55:24.591981 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.591599 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e9cd74e-297a-4671-aeca-8affa476c1c0-extensions-socket-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:24.591981 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:24.591630 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96v7l\" (UniqueName: \"kubernetes.io/projected/3e9cd74e-297a-4671-aeca-8affa476c1c0-kube-api-access-96v7l\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:25.283126 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.283084 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" event={"ID":"7b483439-6180-459d-ace3-e615fe9541de","Type":"ContainerStarted","Data":"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791"} Jun 11 16:55:25.283569 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.283144 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" event={"ID":"7b483439-6180-459d-ace3-e615fe9541de","Type":"ContainerStarted","Data":"bb91fa86be5ff7e36f169ff82c01abdc39a46a42abd7b7423c65d1d3d5cc3901"} Jun 11 16:55:25.283569 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.283169 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:25.284845 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.284821 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" event={"ID":"20a8d8a5-cda8-440c-b039-fa3aab9c9627","Type":"ContainerStarted","Data":"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c"} Jun 11 16:55:25.284945 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.284866 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" event={"ID":"20a8d8a5-cda8-440c-b039-fa3aab9c9627","Type":"ContainerStarted","Data":"5ba20ed6d9d04bf8f02f06ec836eadc3432610f1d04ee2acc1ee3e5859c409fd"} Jun 11 16:55:25.284945 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.284888 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:25.286160 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.286143 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:25.286248 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.286176 2557 scope.go:117] "RemoveContainer" containerID="11bf0dd948446a2d1c28560e058d886e01161e6c5a1028ef99d22dbe26bc5936" Jun 11 16:55:25.287697 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.287676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" event={"ID":"cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd","Type":"ContainerStarted","Data":"8108b465a8e20dcae3ecc58f869ca83ddaf098db9c77053449c5c48927dff3bb"} Jun 11 16:55:25.307500 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.307454 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" podStartSLOduration=13.307436245 podStartE2EDuration="13.307436245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:55:25.305903618 +0000 UTC m=+681.195034995" watchObservedRunningTime="2026-06-11 16:55:25.307436245 +0000 UTC m=+681.196567621" Jun 11 16:55:25.326201 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.326145 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jshq4" podStartSLOduration=1.3618942600000001 podStartE2EDuration="24.326124357s" totalImagesPullingTime="22.964230097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:55:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:55:25.325836623 +0000 UTC m=+681.214968001" watchObservedRunningTime="2026-06-11 16:55:25.326124357 +0000 UTC m=+681.215255736" Jun 11 16:55:25.365473 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:25.365416 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" podStartSLOduration=13.36540123 podStartE2EDuration="13.36540123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:55:25.364543295 +0000 UTC m=+681.253674673" watchObservedRunningTime="2026-06-11 16:55:25.36540123 +0000 UTC m=+681.254532681" Jun 11 16:55:25.367022 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:25.366996 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zffdn\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="3e9cd74e-297a-4671-aeca-8affa476c1c0" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zffdn" Jun 11 16:55:36.297080 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.297052 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:36.297637 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.297204 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 16:55:36.383949 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.383915 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:36.384159 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.384133 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" podUID="7b483439-6180-459d-ace3-e615fe9541de" containerName="manager" containerID="cri-o://2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791" gracePeriod=10 Jun 11 16:55:36.635600 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.635556 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:36.800532 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.800495 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwznw\" (UniqueName: \"kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw\") pod \"7b483439-6180-459d-ace3-e615fe9541de\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " Jun 11 16:55:36.800728 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.800558 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume\") pod \"7b483439-6180-459d-ace3-e615fe9541de\" (UID: \"7b483439-6180-459d-ace3-e615fe9541de\") " Jun 11 16:55:36.801000 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.800978 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b483439-6180-459d-ace3-e615fe9541de" (UID: "7b483439-6180-459d-ace3-e615fe9541de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:55:36.802772 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.802701 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw" (OuterVolumeSpecName: "kube-api-access-kwznw") pod "7b483439-6180-459d-ace3-e615fe9541de" (UID: "7b483439-6180-459d-ace3-e615fe9541de"). InnerVolumeSpecName "kube-api-access-kwznw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:55:36.902151 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.902116 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwznw\" (UniqueName: \"kubernetes.io/projected/7b483439-6180-459d-ace3-e615fe9541de-kube-api-access-kwznw\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:36.902151 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:36.902145 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b483439-6180-459d-ace3-e615fe9541de-extensions-socket-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:55:37.352523 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.352484 2557 generic.go:356] "Generic (PLEG): container finished" podID="7b483439-6180-459d-ace3-e615fe9541de" containerID="2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791" exitCode=0 Jun 11 16:55:37.352523 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.352524 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" Jun 11 16:55:37.353015 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.352567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" event={"ID":"7b483439-6180-459d-ace3-e615fe9541de","Type":"ContainerDied","Data":"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791"} Jun 11 16:55:37.353015 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.352643 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw" event={"ID":"7b483439-6180-459d-ace3-e615fe9541de","Type":"ContainerDied","Data":"bb91fa86be5ff7e36f169ff82c01abdc39a46a42abd7b7423c65d1d3d5cc3901"} Jun 11 16:55:37.353015 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.352675 2557 scope.go:117] "RemoveContainer" containerID="2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791" Jun 11 16:55:37.361806 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.361789 2557 scope.go:117] "RemoveContainer" containerID="2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791" Jun 11 16:55:37.362074 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:55:37.362054 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791\": container with ID starting with 2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791 not found: ID does not exist" containerID="2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791" Jun 11 16:55:37.362143 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.362082 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791"} err="failed to get container status \"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791\": rpc error: code = NotFound desc = could not find container \"2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791\": container with ID starting with 2ffb09936915578581e439e10f519b6db5ec8765b64bd3e64f60ca1df639a791 not found: ID does not exist" Jun 11 16:55:37.375028 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.375006 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:37.381385 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:37.381362 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pq8gw"] Jun 11 16:55:38.517714 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:38.517681 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b483439-6180-459d-ace3-e615fe9541de" path="/var/lib/kubelet/pods/7b483439-6180-459d-ace3-e615fe9541de/volumes" Jun 11 16:55:55.237232 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.237158 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:55:55.237572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.237553 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b483439-6180-459d-ace3-e615fe9541de" containerName="manager" Jun 11 16:55:55.237572 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.237567 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b483439-6180-459d-ace3-e615fe9541de" containerName="manager" Jun 11 16:55:55.237679 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.237652 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b483439-6180-459d-ace3-e615fe9541de" containerName="manager" Jun 11 16:55:55.307739 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.307698 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:55:55.307915 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.307801 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:55:55.311697 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.311673 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g7tzh\"" Jun 11 16:55:55.365738 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.365709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhfbs\" (UniqueName: \"kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs\") pod \"authorino-58cd955675-ckfv8\" (UID: \"1301e17f-01e5-4b65-97bd-67a7e4820e1e\") " pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:55:55.466590 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.466555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhfbs\" (UniqueName: \"kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs\") pod \"authorino-58cd955675-ckfv8\" (UID: \"1301e17f-01e5-4b65-97bd-67a7e4820e1e\") " pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:55:55.478326 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.478291 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhfbs\" (UniqueName: \"kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs\") pod \"authorino-58cd955675-ckfv8\" (UID: \"1301e17f-01e5-4b65-97bd-67a7e4820e1e\") " pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:55:55.527267 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.527189 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:55:55.530359 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.530341 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:55:55.540130 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.540106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:55:55.617631 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.617602 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:55:55.668651 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.668568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9tdw\" (UniqueName: \"kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw\") pod \"authorino-75d556bd77-5djbz\" (UID: \"869184b0-e689-483d-a37a-870772e9ae77\") " pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:55:55.751396 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.751370 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:55:55.754172 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:55:55.754145 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1301e17f_01e5_4b65_97bd_67a7e4820e1e.slice/crio-e31a2db2299859047c9b96cb5576bd75b7ea61aa41727103da519fbdacd81f7e WatchSource:0}: Error finding container e31a2db2299859047c9b96cb5576bd75b7ea61aa41727103da519fbdacd81f7e: Status 404 returned error can't find the container with id e31a2db2299859047c9b96cb5576bd75b7ea61aa41727103da519fbdacd81f7e Jun 11 16:55:55.769894 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.769868 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9tdw\" (UniqueName: \"kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw\") pod \"authorino-75d556bd77-5djbz\" (UID: \"869184b0-e689-483d-a37a-870772e9ae77\") " pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:55:55.780513 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.780441 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9tdw\" (UniqueName: \"kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw\") pod \"authorino-75d556bd77-5djbz\" (UID: \"869184b0-e689-483d-a37a-870772e9ae77\") " pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:55:55.840702 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.840678 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:55:55.971624 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:55.971599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:55:55.973448 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:55:55.973420 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod869184b0_e689_483d_a37a_870772e9ae77.slice/crio-8cc456deb6cdcf3c4fc5ee2353326b2fc90ed9e9360fdfc1a1c3e1438fecf0f8 WatchSource:0}: Error finding container 8cc456deb6cdcf3c4fc5ee2353326b2fc90ed9e9360fdfc1a1c3e1438fecf0f8: Status 404 returned error can't find the container with id 8cc456deb6cdcf3c4fc5ee2353326b2fc90ed9e9360fdfc1a1c3e1438fecf0f8 Jun 11 16:55:56.466796 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:56.466755 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5djbz" event={"ID":"869184b0-e689-483d-a37a-870772e9ae77","Type":"ContainerStarted","Data":"8cc456deb6cdcf3c4fc5ee2353326b2fc90ed9e9360fdfc1a1c3e1438fecf0f8"} Jun 11 16:55:56.469019 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:56.468986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ckfv8" event={"ID":"1301e17f-01e5-4b65-97bd-67a7e4820e1e","Type":"ContainerStarted","Data":"e31a2db2299859047c9b96cb5576bd75b7ea61aa41727103da519fbdacd81f7e"} Jun 11 16:55:59.489662 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:59.489623 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5djbz" event={"ID":"869184b0-e689-483d-a37a-870772e9ae77","Type":"ContainerStarted","Data":"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef"} Jun 11 16:55:59.491020 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:59.490994 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ckfv8" event={"ID":"1301e17f-01e5-4b65-97bd-67a7e4820e1e","Type":"ContainerStarted","Data":"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6"} Jun 11 16:55:59.507999 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:59.507905 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-5djbz" podStartSLOduration=1.239302799 podStartE2EDuration="4.507890213s" totalImagesPullingTime="3.268587414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:55:59.505700435 +0000 UTC m=+715.394831812" watchObservedRunningTime="2026-06-11 16:55:59.507890213 +0000 UTC m=+715.397021568" Jun 11 16:55:59.522235 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:59.522184 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ckfv8" podStartSLOduration=1.023287734 podStartE2EDuration="4.522167069s" totalImagesPullingTime="3.498879335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:55:59.520710939 +0000 UTC m=+715.409842317" watchObservedRunningTime="2026-06-11 16:55:59.522167069 +0000 UTC m=+715.411298481" Jun 11 16:55:59.546452 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:55:59.546417 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:56:01.500102 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:01.500064 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ckfv8" podUID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" containerName="authorino" containerID="cri-o://5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6" gracePeriod=30 Jun 11 16:56:01.754974 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:01.754904 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:56:01.827373 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:01.827340 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhfbs\" (UniqueName: \"kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs\") pod \"1301e17f-01e5-4b65-97bd-67a7e4820e1e\" (UID: \"1301e17f-01e5-4b65-97bd-67a7e4820e1e\") " Jun 11 16:56:01.829473 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:01.829445 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs" (OuterVolumeSpecName: "kube-api-access-bhfbs") pod "1301e17f-01e5-4b65-97bd-67a7e4820e1e" (UID: "1301e17f-01e5-4b65-97bd-67a7e4820e1e"). InnerVolumeSpecName "kube-api-access-bhfbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:56:01.927993 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:01.927955 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhfbs\" (UniqueName: \"kubernetes.io/projected/1301e17f-01e5-4b65-97bd-67a7e4820e1e-kube-api-access-bhfbs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:56:02.505643 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.505606 2557 generic.go:356] "Generic (PLEG): container finished" podID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" containerID="5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6" exitCode=0 Jun 11 16:56:02.506037 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.505657 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ckfv8" Jun 11 16:56:02.506037 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.505698 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ckfv8" event={"ID":"1301e17f-01e5-4b65-97bd-67a7e4820e1e","Type":"ContainerDied","Data":"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6"} Jun 11 16:56:02.506037 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.505745 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ckfv8" event={"ID":"1301e17f-01e5-4b65-97bd-67a7e4820e1e","Type":"ContainerDied","Data":"e31a2db2299859047c9b96cb5576bd75b7ea61aa41727103da519fbdacd81f7e"} Jun 11 16:56:02.506037 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.505770 2557 scope.go:117] "RemoveContainer" containerID="5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6" Jun 11 16:56:02.515446 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.515423 2557 scope.go:117] "RemoveContainer" containerID="5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6" Jun 11 16:56:02.515756 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:56:02.515737 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6\": container with ID starting with 5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6 not found: ID does not exist" containerID="5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6" Jun 11 16:56:02.515837 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.515765 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6"} err="failed to get container status \"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6\": rpc error: code = NotFound desc = could not find container \"5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6\": container with ID starting with 5ab24deddc0a8384807239b00545ab8cd996c3bdce0f3b3decb2c37bcc4c53c6 not found: ID does not exist" Jun 11 16:56:02.526869 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.526807 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:56:02.529355 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:02.529329 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ckfv8"] Jun 11 16:56:04.517553 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:04.517522 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" path="/var/lib/kubelet/pods/1301e17f-01e5-4b65-97bd-67a7e4820e1e/volumes" Jun 11 16:56:45.230053 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.230019 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 16:56:45.230458 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.230353 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" containerName="authorino" Jun 11 16:56:45.230458 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.230364 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" containerName="authorino" Jun 11 16:56:45.230458 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.230439 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1301e17f-01e5-4b65-97bd-67a7e4820e1e" containerName="authorino" Jun 11 16:56:45.235644 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.235622 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:45.238771 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.238745 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-sp72r\"" Jun 11 16:56:45.239476 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.239457 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 16:56:45.239559 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.239519 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 16:56:45.239736 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.239722 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 16:56:45.244660 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.244638 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 16:56:45.307282 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.307242 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtvls\" (UniqueName: \"kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls\") pod \"maas-keycloak-0\" (UID: \"4b8023c9-b316-4e88-9119-d8c745732804\") " pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:45.408690 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.408656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtvls\" (UniqueName: \"kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls\") pod \"maas-keycloak-0\" (UID: \"4b8023c9-b316-4e88-9119-d8c745732804\") " pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:45.419595 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.419553 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtvls\" (UniqueName: \"kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls\") pod \"maas-keycloak-0\" (UID: \"4b8023c9-b316-4e88-9119-d8c745732804\") " pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:45.546444 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.546356 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:45.742619 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:45.742596 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 16:56:45.744524 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:56:45.744498 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8023c9_b316_4e88_9119_d8c745732804.slice/crio-72d832474feab1854dac311ad0e421064bf2d68912b9e305ecdcb77478fa80ef WatchSource:0}: Error finding container 72d832474feab1854dac311ad0e421064bf2d68912b9e305ecdcb77478fa80ef: Status 404 returned error can't find the container with id 72d832474feab1854dac311ad0e421064bf2d68912b9e305ecdcb77478fa80ef Jun 11 16:56:46.708085 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:46.708032 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4b8023c9-b316-4e88-9119-d8c745732804","Type":"ContainerStarted","Data":"72d832474feab1854dac311ad0e421064bf2d68912b9e305ecdcb77478fa80ef"} Jun 11 16:56:52.745036 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:52.744995 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4b8023c9-b316-4e88-9119-d8c745732804","Type":"ContainerStarted","Data":"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb"} Jun 11 16:56:52.763993 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:52.763898 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.5233269379999999 podStartE2EDuration="7.763881588s" totalImagesPullingTime="6.24055465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:56:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:56:52.761837164 +0000 UTC m=+768.650968542" watchObservedRunningTime="2026-06-11 16:56:52.763881588 +0000 UTC m=+768.653013008" Jun 11 16:56:53.547313 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:53.547266 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:53.549289 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:53.549245 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:54.546943 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:54.546890 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:55.546841 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:55.546784 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 16:56:55.547176 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:55.547134 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:56.547088 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:56.547041 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:57.547146 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:57.547075 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:58.548465 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:58.547731 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:56:59.547898 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:56:59.547839 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:00.547311 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:00.547274 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:01.547736 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:01.547679 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:02.547671 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:02.547623 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:03.546921 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:03.546869 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:04.547844 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:04.547798 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 11 16:57:05.657089 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:05.657050 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 16:57:05.672087 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:05.672037 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:57:15.663262 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:15.663224 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 16:57:16.552693 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.552663 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qcrt2"] Jun 11 16:57:16.560967 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.560945 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.563974 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.563947 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qcrt2"] Jun 11 16:57:16.732995 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.732961 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlwwn\" (UniqueName: \"kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn\") pod \"authorino-5d4b67c446-qcrt2\" (UID: \"daeeff90-cf28-42a2-b7a0-3bbaafaee53a\") " pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.799638 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.799597 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qcrt2"] Jun 11 16:57:16.799926 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:57:16.799901 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-hlwwn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-qcrt2" podUID="daeeff90-cf28-42a2-b7a0-3bbaafaee53a" Jun 11 16:57:16.832671 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.832599 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:16.833845 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.833820 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlwwn\" (UniqueName: \"kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn\") pod \"authorino-5d4b67c446-qcrt2\" (UID: \"daeeff90-cf28-42a2-b7a0-3bbaafaee53a\") " pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.834908 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.834893 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:16.849251 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.849226 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:16.859368 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.859339 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlwwn\" (UniqueName: \"kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn\") pod \"authorino-5d4b67c446-qcrt2\" (UID: \"daeeff90-cf28-42a2-b7a0-3bbaafaee53a\") " pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.887935 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.887908 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.894282 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.894255 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:16.934755 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:16.934723 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrntj\" (UniqueName: \"kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj\") pod \"authorino-58fcd4855c-rv65s\" (UID: \"20cfee5c-3664-4cf1-bd67-570c77f46162\") " pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:17.035175 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.035125 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlwwn\" (UniqueName: \"kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn\") pod \"daeeff90-cf28-42a2-b7a0-3bbaafaee53a\" (UID: \"daeeff90-cf28-42a2-b7a0-3bbaafaee53a\") " Jun 11 16:57:17.035362 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.035243 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrntj\" (UniqueName: \"kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj\") pod \"authorino-58fcd4855c-rv65s\" (UID: \"20cfee5c-3664-4cf1-bd67-570c77f46162\") " pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:17.037193 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.037166 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn" (OuterVolumeSpecName: "kube-api-access-hlwwn") pod "daeeff90-cf28-42a2-b7a0-3bbaafaee53a" (UID: "daeeff90-cf28-42a2-b7a0-3bbaafaee53a"). InnerVolumeSpecName "kube-api-access-hlwwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:57:17.045152 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.045129 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrntj\" (UniqueName: \"kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj\") pod \"authorino-58fcd4855c-rv65s\" (UID: \"20cfee5c-3664-4cf1-bd67-570c77f46162\") " pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:17.136114 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.136031 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlwwn\" (UniqueName: \"kubernetes.io/projected/daeeff90-cf28-42a2-b7a0-3bbaafaee53a-kube-api-access-hlwwn\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:57:17.143874 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.143847 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:17.261671 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.261639 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:17.291467 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.291445 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:17.293243 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:57:17.293215 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20cfee5c_3664_4cf1_bd67_570c77f46162.slice/crio-840dc2d88ed2499e0cfc47c1f664c2fcadaa1330872aaef54a29334edc208e53 WatchSource:0}: Error finding container 840dc2d88ed2499e0cfc47c1f664c2fcadaa1330872aaef54a29334edc208e53: Status 404 returned error can't find the container with id 840dc2d88ed2499e0cfc47c1f664c2fcadaa1330872aaef54a29334edc208e53 Jun 11 16:57:17.301621 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.301571 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 16:57:17.304729 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.304713 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.307116 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.307095 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 11 16:57:17.315571 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.315545 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 16:57:17.439984 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.439890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.440155 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.440032 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh2pj\" (UniqueName: \"kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.540854 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.540812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh2pj\" (UniqueName: \"kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.541034 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.540867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.543318 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.543293 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.548867 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.548837 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh2pj\" (UniqueName: \"kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj\") pod \"authorino-5cbf587f7c-t7blz\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.615301 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.615272 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 16:57:17.749775 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.749749 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 16:57:17.752540 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:57:17.752512 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc258bcd0_9fb7_4a73_ad7e_26c72d3fea85.slice/crio-803302b76a9996152e201a982ea53e8661e4831f965d29a41202cf70cd590938 WatchSource:0}: Error finding container 803302b76a9996152e201a982ea53e8661e4831f965d29a41202cf70cd590938: Status 404 returned error can't find the container with id 803302b76a9996152e201a982ea53e8661e4831f965d29a41202cf70cd590938 Jun 11 16:57:17.895186 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.895151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fcd4855c-rv65s" event={"ID":"20cfee5c-3664-4cf1-bd67-570c77f46162","Type":"ContainerStarted","Data":"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff"} Jun 11 16:57:17.895400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.895199 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fcd4855c-rv65s" event={"ID":"20cfee5c-3664-4cf1-bd67-570c77f46162","Type":"ContainerStarted","Data":"840dc2d88ed2499e0cfc47c1f664c2fcadaa1330872aaef54a29334edc208e53"} Jun 11 16:57:17.895400 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.895163 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58fcd4855c-rv65s" podUID="20cfee5c-3664-4cf1-bd67-570c77f46162" containerName="authorino" containerID="cri-o://55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff" gracePeriod=30 Jun 11 16:57:17.896428 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.896401 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qcrt2" Jun 11 16:57:17.896534 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.896440 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" event={"ID":"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85","Type":"ContainerStarted","Data":"803302b76a9996152e201a982ea53e8661e4831f965d29a41202cf70cd590938"} Jun 11 16:57:17.911211 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.911168 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58fcd4855c-rv65s" podStartSLOduration=1.526028607 podStartE2EDuration="1.911153495s" totalImagesPullingTime="385.124888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:57:17.909733952 +0000 UTC m=+793.798865330" watchObservedRunningTime="2026-06-11 16:57:17.911153495 +0000 UTC m=+793.800284871" Jun 11 16:57:17.972727 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.972698 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qcrt2"] Jun 11 16:57:17.974978 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:17.974953 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qcrt2"] Jun 11 16:57:18.137224 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.137203 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:18.247124 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.247093 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrntj\" (UniqueName: \"kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj\") pod \"20cfee5c-3664-4cf1-bd67-570c77f46162\" (UID: \"20cfee5c-3664-4cf1-bd67-570c77f46162\") " Jun 11 16:57:18.249181 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.249155 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj" (OuterVolumeSpecName: "kube-api-access-vrntj") pod "20cfee5c-3664-4cf1-bd67-570c77f46162" (UID: "20cfee5c-3664-4cf1-bd67-570c77f46162"). InnerVolumeSpecName "kube-api-access-vrntj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:57:18.348501 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.348457 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrntj\" (UniqueName: \"kubernetes.io/projected/20cfee5c-3664-4cf1-bd67-570c77f46162-kube-api-access-vrntj\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:57:18.518405 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.518326 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daeeff90-cf28-42a2-b7a0-3bbaafaee53a" path="/var/lib/kubelet/pods/daeeff90-cf28-42a2-b7a0-3bbaafaee53a/volumes" Jun 11 16:57:18.902279 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.902246 2557 generic.go:356] "Generic (PLEG): container finished" podID="20cfee5c-3664-4cf1-bd67-570c77f46162" containerID="55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff" exitCode=0 Jun 11 16:57:18.902758 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.902291 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58fcd4855c-rv65s" Jun 11 16:57:18.902758 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.902294 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fcd4855c-rv65s" event={"ID":"20cfee5c-3664-4cf1-bd67-570c77f46162","Type":"ContainerDied","Data":"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff"} Jun 11 16:57:18.902758 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.902341 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58fcd4855c-rv65s" event={"ID":"20cfee5c-3664-4cf1-bd67-570c77f46162","Type":"ContainerDied","Data":"840dc2d88ed2499e0cfc47c1f664c2fcadaa1330872aaef54a29334edc208e53"} Jun 11 16:57:18.902758 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.902359 2557 scope.go:117] "RemoveContainer" containerID="55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff" Jun 11 16:57:18.904251 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.904221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" event={"ID":"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85","Type":"ContainerStarted","Data":"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3"} Jun 11 16:57:18.911395 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.911370 2557 scope.go:117] "RemoveContainer" containerID="55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff" Jun 11 16:57:18.911699 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:57:18.911679 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff\": container with ID starting with 55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff not found: ID does not exist" containerID="55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff" Jun 11 16:57:18.911773 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.911709 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff"} err="failed to get container status \"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff\": rpc error: code = NotFound desc = could not find container \"55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff\": container with ID starting with 55136a57be606e988efa5428acbaf70a16fc31f9e7cc7caa18269a79389723ff not found: ID does not exist" Jun 11 16:57:18.918262 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.918239 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:18.922727 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.922707 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58fcd4855c-rv65s"] Jun 11 16:57:18.937471 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.937425 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" podStartSLOduration=1.646478557 podStartE2EDuration="1.937408863s" totalImagesPullingTime="290.930306ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:57:18.93571615 +0000 UTC m=+794.824847527" watchObservedRunningTime="2026-06-11 16:57:18.937408863 +0000 UTC m=+794.826540240" Jun 11 16:57:18.966208 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.966171 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:57:18.966403 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:18.966379 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-5djbz" podUID="869184b0-e689-483d-a37a-870772e9ae77" containerName="authorino" containerID="cri-o://0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef" gracePeriod=30 Jun 11 16:57:19.217637 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.217613 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:57:19.358983 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.358952 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9tdw\" (UniqueName: \"kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw\") pod \"869184b0-e689-483d-a37a-870772e9ae77\" (UID: \"869184b0-e689-483d-a37a-870772e9ae77\") " Jun 11 16:57:19.361010 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.360979 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw" (OuterVolumeSpecName: "kube-api-access-g9tdw") pod "869184b0-e689-483d-a37a-870772e9ae77" (UID: "869184b0-e689-483d-a37a-870772e9ae77"). InnerVolumeSpecName "kube-api-access-g9tdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:57:19.460030 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.459948 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9tdw\" (UniqueName: \"kubernetes.io/projected/869184b0-e689-483d-a37a-870772e9ae77-kube-api-access-g9tdw\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:57:19.916962 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.916925 2557 generic.go:356] "Generic (PLEG): container finished" podID="869184b0-e689-483d-a37a-870772e9ae77" containerID="0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef" exitCode=0 Jun 11 16:57:19.917436 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.916975 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5djbz" Jun 11 16:57:19.917436 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.917010 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5djbz" event={"ID":"869184b0-e689-483d-a37a-870772e9ae77","Type":"ContainerDied","Data":"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef"} Jun 11 16:57:19.917436 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.917056 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5djbz" event={"ID":"869184b0-e689-483d-a37a-870772e9ae77","Type":"ContainerDied","Data":"8cc456deb6cdcf3c4fc5ee2353326b2fc90ed9e9360fdfc1a1c3e1438fecf0f8"} Jun 11 16:57:19.917436 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.917078 2557 scope.go:117] "RemoveContainer" containerID="0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef" Jun 11 16:57:19.927280 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.927256 2557 scope.go:117] "RemoveContainer" containerID="0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef" Jun 11 16:57:19.927838 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:57:19.927813 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef\": container with ID starting with 0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef not found: ID does not exist" containerID="0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef" Jun 11 16:57:19.927938 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.927846 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef"} err="failed to get container status \"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef\": rpc error: code = NotFound desc = could not find container \"0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef\": container with ID starting with 0d4f6beb4220d8e94746cf3e121d46207bb55bae54f4f39e99b81c9af9b003ef not found: ID does not exist" Jun 11 16:57:19.942006 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.941985 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:57:19.945313 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:19.945288 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5djbz"] Jun 11 16:57:20.517263 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:20.517229 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20cfee5c-3664-4cf1-bd67-570c77f46162" path="/var/lib/kubelet/pods/20cfee5c-3664-4cf1-bd67-570c77f46162/volumes" Jun 11 16:57:20.517594 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:20.517562 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="869184b0-e689-483d-a37a-870772e9ae77" path="/var/lib/kubelet/pods/869184b0-e689-483d-a37a-870772e9ae77/volumes" Jun 11 16:57:22.682017 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.681982 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:22.682383 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682357 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20cfee5c-3664-4cf1-bd67-570c77f46162" containerName="authorino" Jun 11 16:57:22.682383 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682368 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="20cfee5c-3664-4cf1-bd67-570c77f46162" containerName="authorino" Jun 11 16:57:22.682383 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682381 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="869184b0-e689-483d-a37a-870772e9ae77" containerName="authorino" Jun 11 16:57:22.682486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682387 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="869184b0-e689-483d-a37a-870772e9ae77" containerName="authorino" Jun 11 16:57:22.682486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682443 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="20cfee5c-3664-4cf1-bd67-570c77f46162" containerName="authorino" Jun 11 16:57:22.682486 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.682453 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="869184b0-e689-483d-a37a-870772e9ae77" containerName="authorino" Jun 11 16:57:22.684305 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.684290 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.687293 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.687265 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 16:57:22.687415 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.687309 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 16:57:22.688026 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.688009 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pzdvc\"" Jun 11 16:57:22.695342 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.695322 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:22.793905 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.793873 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvtvx\" (UniqueName: \"kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.794066 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.793922 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.895411 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.895376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvtvx\" (UniqueName: \"kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.895597 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.895419 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.897774 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.897752 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.904310 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.904287 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvtvx\" (UniqueName: \"kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx\") pod \"maas-controller-6bf88897cd-96r42\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:22.995318 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:22.995291 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:23.337203 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:23.337106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:23.340062 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:57:23.340028 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37269f01_f49e_4d16_95b1_0087343a0a4d.slice/crio-7834d8dff92cd59613762f74a95a96c67a9f960330be092d1617b7361d2c405e WatchSource:0}: Error finding container 7834d8dff92cd59613762f74a95a96c67a9f960330be092d1617b7361d2c405e: Status 404 returned error can't find the container with id 7834d8dff92cd59613762f74a95a96c67a9f960330be092d1617b7361d2c405e Jun 11 16:57:23.941920 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:23.941831 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf88897cd-96r42" event={"ID":"37269f01-f49e-4d16-95b1-0087343a0a4d","Type":"ContainerStarted","Data":"7834d8dff92cd59613762f74a95a96c67a9f960330be092d1617b7361d2c405e"} Jun 11 16:57:25.954994 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:25.954901 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf88897cd-96r42" event={"ID":"37269f01-f49e-4d16-95b1-0087343a0a4d","Type":"ContainerStarted","Data":"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71"} Jun 11 16:57:25.955406 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:25.955160 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:26.024176 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:26.024106 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6bf88897cd-96r42" podStartSLOduration=1.678432766 podStartE2EDuration="4.024085532s" totalImagesPullingTime="2.345652766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:57:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:57:26.016034631 +0000 UTC m=+801.905166008" watchObservedRunningTime="2026-06-11 16:57:26.024085532 +0000 UTC m=+801.913216912" Jun 11 16:57:36.965662 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:36.965631 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:37.344783 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.344705 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:37.344923 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.344876 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6bf88897cd-96r42" podUID="37269f01-f49e-4d16-95b1-0087343a0a4d" containerName="manager" containerID="cri-o://027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71" gracePeriod=30 Jun 11 16:57:37.586926 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.586897 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:37.630509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.630425 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvtvx\" (UniqueName: \"kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx\") pod \"37269f01-f49e-4d16-95b1-0087343a0a4d\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " Jun 11 16:57:37.630509 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.630472 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert\") pod \"37269f01-f49e-4d16-95b1-0087343a0a4d\" (UID: \"37269f01-f49e-4d16-95b1-0087343a0a4d\") " Jun 11 16:57:37.632592 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.632558 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "37269f01-f49e-4d16-95b1-0087343a0a4d" (UID: "37269f01-f49e-4d16-95b1-0087343a0a4d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:57:37.632680 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.632655 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx" (OuterVolumeSpecName: "kube-api-access-nvtvx") pod "37269f01-f49e-4d16-95b1-0087343a0a4d" (UID: "37269f01-f49e-4d16-95b1-0087343a0a4d"). InnerVolumeSpecName "kube-api-access-nvtvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:57:37.732079 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.732045 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvtvx\" (UniqueName: \"kubernetes.io/projected/37269f01-f49e-4d16-95b1-0087343a0a4d-kube-api-access-nvtvx\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:57:37.732079 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:37.732074 2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37269f01-f49e-4d16-95b1-0087343a0a4d-webhook-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 16:57:38.014703 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.014668 2557 generic.go:356] "Generic (PLEG): container finished" podID="37269f01-f49e-4d16-95b1-0087343a0a4d" containerID="027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71" exitCode=0 Jun 11 16:57:38.015090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.014728 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6bf88897cd-96r42" Jun 11 16:57:38.015090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.014761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf88897cd-96r42" event={"ID":"37269f01-f49e-4d16-95b1-0087343a0a4d","Type":"ContainerDied","Data":"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71"} Jun 11 16:57:38.015090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.014833 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6bf88897cd-96r42" event={"ID":"37269f01-f49e-4d16-95b1-0087343a0a4d","Type":"ContainerDied","Data":"7834d8dff92cd59613762f74a95a96c67a9f960330be092d1617b7361d2c405e"} Jun 11 16:57:38.015090 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.014858 2557 scope.go:117] "RemoveContainer" containerID="027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71" Jun 11 16:57:38.025061 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.025043 2557 scope.go:117] "RemoveContainer" containerID="027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71" Jun 11 16:57:38.025334 ip-10-0-128-197 kubenswrapper[2557]: E0611 16:57:38.025314 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71\": container with ID starting with 027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71 not found: ID does not exist" containerID="027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71" Jun 11 16:57:38.025387 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.025346 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71"} err="failed to get container status \"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71\": rpc error: code = NotFound desc = could not find container \"027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71\": container with ID starting with 027752f2db97bc928ed78ced984bb0624a4d43a84d322ff301f4053af41ebb71 not found: ID does not exist" Jun 11 16:57:38.038238 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.038203 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:38.043471 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.043449 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6bf88897cd-96r42"] Jun 11 16:57:38.518083 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:38.518050 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37269f01-f49e-4d16-95b1-0087343a0a4d" path="/var/lib/kubelet/pods/37269f01-f49e-4d16-95b1-0087343a0a4d/volumes" Jun 11 16:57:55.118182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.118145 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5bc764bbcb-9mrjr"] Jun 11 16:57:55.118683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.118534 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37269f01-f49e-4d16-95b1-0087343a0a4d" containerName="manager" Jun 11 16:57:55.118683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.118546 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="37269f01-f49e-4d16-95b1-0087343a0a4d" containerName="manager" Jun 11 16:57:55.118683 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.118630 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="37269f01-f49e-4d16-95b1-0087343a0a4d" containerName="manager" Jun 11 16:57:55.124612 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.124572 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.127050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.127029 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-v6xgm\"" Jun 11 16:57:55.127196 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.127060 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 11 16:57:55.136194 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.136161 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5bc764bbcb-9mrjr"] Jun 11 16:57:55.192182 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.192147 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4433523c-0cf8-46ca-ba3b-046e53deab54-maas-api-tls\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.192366 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.192195 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gjcm\" (UniqueName: \"kubernetes.io/projected/4433523c-0cf8-46ca-ba3b-046e53deab54-kube-api-access-5gjcm\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.192366 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.192269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4433523c-0cf8-46ca-ba3b-046e53deab54-openshift-service-ca\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.293724 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.293690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4433523c-0cf8-46ca-ba3b-046e53deab54-openshift-service-ca\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.293926 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.293745 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4433523c-0cf8-46ca-ba3b-046e53deab54-maas-api-tls\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.293926 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.293777 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gjcm\" (UniqueName: \"kubernetes.io/projected/4433523c-0cf8-46ca-ba3b-046e53deab54-kube-api-access-5gjcm\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.294368 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.294346 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4433523c-0cf8-46ca-ba3b-046e53deab54-openshift-service-ca\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.296189 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.296169 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4433523c-0cf8-46ca-ba3b-046e53deab54-maas-api-tls\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.306651 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.306627 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gjcm\" (UniqueName: \"kubernetes.io/projected/4433523c-0cf8-46ca-ba3b-046e53deab54-kube-api-access-5gjcm\") pod \"maas-api-5bc764bbcb-9mrjr\" (UID: \"4433523c-0cf8-46ca-ba3b-046e53deab54\") " pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.435169 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.435095 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:55.702447 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.702356 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-h64r7"] Jun 11 16:57:55.707641 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.707620 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:55.710466 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.710447 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 16:57:55.729804 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.729783 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-h64r7"] Jun 11 16:57:55.793246 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.793216 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5bc764bbcb-9mrjr"] Jun 11 16:57:55.795931 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:57:55.795899 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4433523c_0cf8_46ca_ba3b_046e53deab54.slice/crio-25e767e736b182df9dc84f0e8a4504f655a116d088d1e96c16cb9b2a00b4a9c8 WatchSource:0}: Error finding container 25e767e736b182df9dc84f0e8a4504f655a116d088d1e96c16cb9b2a00b4a9c8: Status 404 returned error can't find the container with id 25e767e736b182df9dc84f0e8a4504f655a116d088d1e96c16cb9b2a00b4a9c8 Jun 11 16:57:55.798093 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.798056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzgbm\" (UniqueName: \"kubernetes.io/projected/1536f481-6401-4ea1-b14a-b52fee53af2a-kube-api-access-dzgbm\") pod \"payload-pre-processing-579785bc64-h64r7\" (UID: \"1536f481-6401-4ea1-b14a-b52fee53af2a\") " pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:55.899061 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.899026 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzgbm\" (UniqueName: \"kubernetes.io/projected/1536f481-6401-4ea1-b14a-b52fee53af2a-kube-api-access-dzgbm\") pod \"payload-pre-processing-579785bc64-h64r7\" (UID: \"1536f481-6401-4ea1-b14a-b52fee53af2a\") " pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:55.908251 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:55.908224 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzgbm\" (UniqueName: \"kubernetes.io/projected/1536f481-6401-4ea1-b14a-b52fee53af2a-kube-api-access-dzgbm\") pod \"payload-pre-processing-579785bc64-h64r7\" (UID: \"1536f481-6401-4ea1-b14a-b52fee53af2a\") " pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:56.018747 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:56.018716 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:56.101796 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:56.101758 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" event={"ID":"4433523c-0cf8-46ca-ba3b-046e53deab54","Type":"ContainerStarted","Data":"25e767e736b182df9dc84f0e8a4504f655a116d088d1e96c16cb9b2a00b4a9c8"} Jun 11 16:57:56.159646 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:56.159619 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-h64r7"] Jun 11 16:57:56.162325 ip-10-0-128-197 kubenswrapper[2557]: W0611 16:57:56.162296 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1536f481_6401_4ea1_b14a_b52fee53af2a.slice/crio-30cebc74ca1f65aafd600ed1ae55a850fa2e650d7f381f76870e154e9949d2be WatchSource:0}: Error finding container 30cebc74ca1f65aafd600ed1ae55a850fa2e650d7f381f76870e154e9949d2be: Status 404 returned error can't find the container with id 30cebc74ca1f65aafd600ed1ae55a850fa2e650d7f381f76870e154e9949d2be Jun 11 16:57:57.110214 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:57.110176 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" event={"ID":"1536f481-6401-4ea1-b14a-b52fee53af2a","Type":"ContainerStarted","Data":"30cebc74ca1f65aafd600ed1ae55a850fa2e650d7f381f76870e154e9949d2be"} Jun 11 16:57:59.122484 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:59.122441 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" event={"ID":"1536f481-6401-4ea1-b14a-b52fee53af2a","Type":"ContainerStarted","Data":"7a19290232eff2e5f7b08a54d369a95535f214a41268389f3ed6160bcba48ee6"} Jun 11 16:57:59.122980 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:59.122535 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:57:59.123967 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:59.123946 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" event={"ID":"4433523c-0cf8-46ca-ba3b-046e53deab54","Type":"ContainerStarted","Data":"b1a0693c1cf2db636a72bcbf2406e8348c75891b8d7481869721677c8649b45a"} Jun 11 16:57:59.124050 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:59.123995 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:57:59.149484 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:57:59.149436 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" podStartSLOduration=1.708513402 podStartE2EDuration="4.149419312s" totalImagesPullingTime="2.44090591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:57:59.145849302 +0000 UTC m=+835.034980680" watchObservedRunningTime="2026-06-11 16:57:59.149419312 +0000 UTC m=+835.038550688" Jun 11 16:58:05.138460 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:58:05.138432 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" Jun 11 16:58:05.161275 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:58:05.161225 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5bc764bbcb-9mrjr" podStartSLOduration=7.431287161 podStartE2EDuration="10.161209478s" totalImagesPullingTime="2.729922317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:57:59.191022513 +0000 UTC m=+835.080153890" watchObservedRunningTime="2026-06-11 16:58:05.161209478 +0000 UTC m=+841.050340855" Jun 11 16:58:10.136140 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:58:10.136109 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-h64r7" Jun 11 16:59:04.610207 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:59:04.610110 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 16:59:04.616114 ip-10-0-128-197 kubenswrapper[2557]: I0611 16:59:04.614705 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:00:09.527367 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:09.527319 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:09.527892 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:09.527553 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" containerID="cri-o://9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb" gracePeriod=30 Jun 11 17:00:11.573722 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.573697 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:11.726780 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.726751 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtvls\" (UniqueName: \"kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls\") pod \"4b8023c9-b316-4e88-9119-d8c745732804\" (UID: \"4b8023c9-b316-4e88-9119-d8c745732804\") " Jun 11 17:00:11.728940 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.728914 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls" (OuterVolumeSpecName: "kube-api-access-xtvls") pod "4b8023c9-b316-4e88-9119-d8c745732804" (UID: "4b8023c9-b316-4e88-9119-d8c745732804"). InnerVolumeSpecName "kube-api-access-xtvls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 17:00:11.751547 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.751513 2557 generic.go:356] "Generic (PLEG): container finished" podID="4b8023c9-b316-4e88-9119-d8c745732804" containerID="9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb" exitCode=143 Jun 11 17:00:11.751717 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.751567 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:11.751717 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.751611 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4b8023c9-b316-4e88-9119-d8c745732804","Type":"ContainerDied","Data":"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb"} Jun 11 17:00:11.751717 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.751668 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"4b8023c9-b316-4e88-9119-d8c745732804","Type":"ContainerDied","Data":"72d832474feab1854dac311ad0e421064bf2d68912b9e305ecdcb77478fa80ef"} Jun 11 17:00:11.751717 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.751689 2557 scope.go:117] "RemoveContainer" containerID="9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb" Jun 11 17:00:11.762059 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.762040 2557 scope.go:117] "RemoveContainer" containerID="9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb" Jun 11 17:00:11.762409 ip-10-0-128-197 kubenswrapper[2557]: E0611 17:00:11.762379 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb\": container with ID starting with 9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb not found: ID does not exist" containerID="9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb" Jun 11 17:00:11.762484 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.762410 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb"} err="failed to get container status \"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb\": rpc error: code = NotFound desc = could not find container \"9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb\": container with ID starting with 9b532f147fd2c899687b55be46c2db0d71dd0a98ef919a5eaf60246ac30863bb not found: ID does not exist" Jun 11 17:00:11.778000 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.777976 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:11.783951 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.783930 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:11.815470 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.815434 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:11.815826 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.815812 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" Jun 11 17:00:11.815880 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.815827 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" Jun 11 17:00:11.815915 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.815901 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b8023c9-b316-4e88-9119-d8c745732804" containerName="keycloak" Jun 11 17:00:11.820463 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.820447 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:11.823047 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.823026 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 17:00:11.823047 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.823056 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 17:00:11.823208 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.823184 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 11 17:00:11.823394 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.823380 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-sp72r\"" Jun 11 17:00:11.823441 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.823389 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 11 17:00:11.827503 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.827481 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtvls\" (UniqueName: \"kubernetes.io/projected/4b8023c9-b316-4e88-9119-d8c745732804-kube-api-access-xtvls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:00:11.847225 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.847196 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:11.928095 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.928052 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d951208a-9f3b-4a04-b80d-13f2f0b824fd-test-realms\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:11.928278 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:11.928138 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h6qz\" (UniqueName: \"kubernetes.io/projected/d951208a-9f3b-4a04-b80d-13f2f0b824fd-kube-api-access-7h6qz\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.028998 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.028891 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d951208a-9f3b-4a04-b80d-13f2f0b824fd-test-realms\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.029157 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.029010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h6qz\" (UniqueName: \"kubernetes.io/projected/d951208a-9f3b-4a04-b80d-13f2f0b824fd-kube-api-access-7h6qz\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.029644 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.029627 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/d951208a-9f3b-4a04-b80d-13f2f0b824fd-test-realms\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.041036 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.041014 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h6qz\" (UniqueName: \"kubernetes.io/projected/d951208a-9f3b-4a04-b80d-13f2f0b824fd-kube-api-access-7h6qz\") pod \"maas-keycloak-0\" (UID: \"d951208a-9f3b-4a04-b80d-13f2f0b824fd\") " pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.130317 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.130264 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:12.264969 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.264945 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 11 17:00:12.267257 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:00:12.267226 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd951208a_9f3b_4a04_b80d_13f2f0b824fd.slice/crio-fe054141ccc49fe295a82f385652318f09b577b49dfa0f70b4ed00e63c4b6529 WatchSource:0}: Error finding container fe054141ccc49fe295a82f385652318f09b577b49dfa0f70b4ed00e63c4b6529: Status 404 returned error can't find the container with id fe054141ccc49fe295a82f385652318f09b577b49dfa0f70b4ed00e63c4b6529 Jun 11 17:00:12.269052 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.269033 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 17:00:12.517809 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.517778 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b8023c9-b316-4e88-9119-d8c745732804" path="/var/lib/kubelet/pods/4b8023c9-b316-4e88-9119-d8c745732804/volumes" Jun 11 17:00:12.759177 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.759146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d951208a-9f3b-4a04-b80d-13f2f0b824fd","Type":"ContainerStarted","Data":"1fb30f724af61dabc81cae8a3214c3a0bd92d8e6b156a284ffa4699bbaba8d0c"} Jun 11 17:00:12.759621 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.759200 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"d951208a-9f3b-4a04-b80d-13f2f0b824fd","Type":"ContainerStarted","Data":"fe054141ccc49fe295a82f385652318f09b577b49dfa0f70b4ed00e63c4b6529"} Jun 11 17:00:12.786202 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:12.786097 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.387153803 podStartE2EDuration="1.786082312s" totalImagesPullingTime="398.928509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:00:12.784924527 +0000 UTC m=+968.674055904" watchObservedRunningTime="2026-06-11 17:00:12.786082312 +0000 UTC m=+968.675213689" Jun 11 17:00:13.131080 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:13.130992 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:13.132807 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:13.132771 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:14.131159 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:14.131096 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:15.131105 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:15.131052 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:16.131363 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:16.131312 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:17.131508 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:17.131456 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:18.131548 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:18.131500 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:19.131326 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:19.131206 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:20.131425 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:20.131366 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:21.131198 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:21.131151 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:22.130615 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:22.130536 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:22.131105 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:22.130858 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:23.131036 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:23.130986 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:24.130958 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:24.130910 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:25.130979 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:25.130935 2557 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.43:9000/health/started\": dial tcp 10.132.0.43:9000: connect: connection refused" Jun 11 17:00:26.254676 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:26.254642 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:26.274034 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:26.273988 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="d951208a-9f3b-4a04-b80d-13f2f0b824fd" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 17:00:36.259944 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:36.259905 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 11 17:00:46.974449 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:46.974413 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:00:46.979857 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:46.979840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:46.982236 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:46.982214 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 11 17:00:46.990466 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:46.990436 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:00:47.108689 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.108646 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll4g2\" (UniqueName: \"kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.108870 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.108704 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.108870 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.108825 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.209498 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.209456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll4g2\" (UniqueName: \"kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.209714 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.209508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.209714 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.209593 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.210291 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.210274 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.211914 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.211895 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.220597 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.220552 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll4g2\" (UniqueName: \"kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2\") pod \"authorino-5676d68769-r94jq\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.289350 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.289271 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:00:47.430869 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.430840 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:00:47.432609 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:00:47.432566 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9770912d_4f93_4e9a_8e4a_e8d5b9f8e5c2.slice/crio-c6837f60ae18049244371930f67325976af7780b8fb2f6198e007dec6322d997 WatchSource:0}: Error finding container c6837f60ae18049244371930f67325976af7780b8fb2f6198e007dec6322d997: Status 404 returned error can't find the container with id c6837f60ae18049244371930f67325976af7780b8fb2f6198e007dec6322d997 Jun 11 17:00:47.970392 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.970354 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5676d68769-r94jq" event={"ID":"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2","Type":"ContainerStarted","Data":"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc"} Jun 11 17:00:47.970392 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.970399 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5676d68769-r94jq" event={"ID":"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2","Type":"ContainerStarted","Data":"c6837f60ae18049244371930f67325976af7780b8fb2f6198e007dec6322d997"} Jun 11 17:00:47.985163 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:47.985109 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5676d68769-r94jq" podStartSLOduration=1.596243593 podStartE2EDuration="1.985090307s" totalImagesPullingTime="388.846714ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:00:47.984280271 +0000 UTC m=+1003.873411649" watchObservedRunningTime="2026-06-11 17:00:47.985090307 +0000 UTC m=+1003.874221714" Jun 11 17:00:48.013250 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.013214 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 17:00:48.013462 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.013429 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" podUID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" containerName="authorino" containerID="cri-o://ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3" gracePeriod=30 Jun 11 17:00:48.271780 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.271756 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 17:00:48.423108 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.423010 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert\") pod \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " Jun 11 17:00:48.423108 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.423091 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh2pj\" (UniqueName: \"kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj\") pod \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\" (UID: \"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85\") " Jun 11 17:00:48.425094 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.425070 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj" (OuterVolumeSpecName: "kube-api-access-dh2pj") pod "c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" (UID: "c258bcd0-9fb7-4a73-ad7e-26c72d3fea85"). InnerVolumeSpecName "kube-api-access-dh2pj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 17:00:48.434858 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.434836 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" (UID: "c258bcd0-9fb7-4a73-ad7e-26c72d3fea85"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 17:00:48.523702 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.523665 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh2pj\" (UniqueName: \"kubernetes.io/projected/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-kube-api-access-dh2pj\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:00:48.523702 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.523690 2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85-tls-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:00:48.976192 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.976156 2557 generic.go:356] "Generic (PLEG): container finished" podID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" containerID="ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3" exitCode=0 Jun 11 17:00:48.976421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.976202 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" Jun 11 17:00:48.976421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.976237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" event={"ID":"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85","Type":"ContainerDied","Data":"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3"} Jun 11 17:00:48.976421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.976279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cbf587f7c-t7blz" event={"ID":"c258bcd0-9fb7-4a73-ad7e-26c72d3fea85","Type":"ContainerDied","Data":"803302b76a9996152e201a982ea53e8661e4831f965d29a41202cf70cd590938"} Jun 11 17:00:48.976421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.976296 2557 scope.go:117] "RemoveContainer" containerID="ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3" Jun 11 17:00:48.985027 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.985013 2557 scope.go:117] "RemoveContainer" containerID="ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3" Jun 11 17:00:48.985281 ip-10-0-128-197 kubenswrapper[2557]: E0611 17:00:48.985262 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3\": container with ID starting with ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3 not found: ID does not exist" containerID="ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3" Jun 11 17:00:48.985515 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.985292 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3"} err="failed to get container status \"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3\": rpc error: code = NotFound desc = could not find container \"ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3\": container with ID starting with ba8a90f2f3ecef8945c91e430189d7599035ca106f30982f206a0e5f144822a3 not found: ID does not exist" Jun 11 17:00:48.993050 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.993026 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 17:00:48.999527 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:48.999504 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cbf587f7c-t7blz"] Jun 11 17:00:50.518623 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:00:50.518562 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" path="/var/lib/kubelet/pods/c258bcd0-9fb7-4a73-ad7e-26c72d3fea85/volumes" Jun 11 17:01:52.942656 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.942564 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x"] Jun 11 17:01:52.943080 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.942955 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" containerName="authorino" Jun 11 17:01:52.943080 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.942966 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" containerName="authorino" Jun 11 17:01:52.943080 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.943041 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c258bcd0-9fb7-4a73-ad7e-26c72d3fea85" containerName="authorino" Jun 11 17:01:52.946386 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.946365 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:52.949255 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.949226 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 17:01:52.949495 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.949477 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 17:01:52.949846 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.949829 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 17:01:52.949945 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.949929 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8fhbj\"" Jun 11 17:01:52.959089 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:52.959066 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x"] Jun 11 17:01:53.003219 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.003346 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003229 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.003346 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003250 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6kfx\" (UniqueName: \"kubernetes.io/projected/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kube-api-access-d6kfx\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.003346 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003331 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6a726e-ecab-4bed-ae85-6e4395d588c1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.003457 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003395 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.003457 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.003425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.104824 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.104787 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6a726e-ecab-4bed-ae85-6e4395d588c1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105009 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.104869 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105009 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.104898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105009 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.104956 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105009 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.104994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105231 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.105015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6kfx\" (UniqueName: \"kubernetes.io/projected/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kube-api-access-d6kfx\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105390 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.105349 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105524 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.105416 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.105524 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.105437 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.107238 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.107220 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f6a726e-ecab-4bed-ae85-6e4395d588c1-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.107526 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.107506 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f6a726e-ecab-4bed-ae85-6e4395d588c1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.114808 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.114779 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6kfx\" (UniqueName: \"kubernetes.io/projected/8f6a726e-ecab-4bed-ae85-6e4395d588c1-kube-api-access-d6kfx\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x\" (UID: \"8f6a726e-ecab-4bed-ae85-6e4395d588c1\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.256778 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.256743 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:01:53.395104 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:53.395077 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x"] Jun 11 17:01:53.397402 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:01:53.397372 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6a726e_ecab_4bed_ae85_6e4395d588c1.slice/crio-adadb08006775e0ab3f0c0195bfd92adc064aa0e7ca761af4ce2b495751f4494 WatchSource:0}: Error finding container adadb08006775e0ab3f0c0195bfd92adc064aa0e7ca761af4ce2b495751f4494: Status 404 returned error can't find the container with id adadb08006775e0ab3f0c0195bfd92adc064aa0e7ca761af4ce2b495751f4494 Jun 11 17:01:54.307809 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:54.307770 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" event={"ID":"8f6a726e-ecab-4bed-ae85-6e4395d588c1","Type":"ContainerStarted","Data":"adadb08006775e0ab3f0c0195bfd92adc064aa0e7ca761af4ce2b495751f4494"} Jun 11 17:01:57.325927 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:01:57.325890 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" event={"ID":"8f6a726e-ecab-4bed-ae85-6e4395d588c1","Type":"ContainerStarted","Data":"24acc48fcf89b103eb9add517046595eb24f213ea70129bd2898a07450be26ca"} Jun 11 17:02:13.400774 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:13.400737 2557 generic.go:356] "Generic (PLEG): container finished" podID="8f6a726e-ecab-4bed-ae85-6e4395d588c1" containerID="24acc48fcf89b103eb9add517046595eb24f213ea70129bd2898a07450be26ca" exitCode=0 Jun 11 17:02:13.401155 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:13.400811 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" event={"ID":"8f6a726e-ecab-4bed-ae85-6e4395d588c1","Type":"ContainerDied","Data":"24acc48fcf89b103eb9add517046595eb24f213ea70129bd2898a07450be26ca"} Jun 11 17:02:15.413909 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:15.413875 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" event={"ID":"8f6a726e-ecab-4bed-ae85-6e4395d588c1","Type":"ContainerStarted","Data":"6f4306109a1b1e9ec3b5b5c3e1bb22cfd9e8787187640851f2e5aeae3633e2fc"} Jun 11 17:02:15.447591 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:15.447528 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" podStartSLOduration=2.610609126 podStartE2EDuration="23.447512381s" totalImagesPullingTime="4.934719083s" totalInitContainerRuntime="15.902184172s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:01:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:02:15.446253045 +0000 UTC m=+1091.335384423" watchObservedRunningTime="2026-06-11 17:02:15.447512381 +0000 UTC m=+1091.336643758" Jun 11 17:02:23.257015 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:23.256975 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:02:23.257392 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:23.257052 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:02:23.269457 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:23.269414 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:02:23.463390 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:02:23.463354 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x" Jun 11 17:03:40.429427 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.429334 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b5b4757b4-42sb5"] Jun 11 17:03:40.432466 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.432443 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.441795 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.441771 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b5b4757b4-42sb5"] Jun 11 17:03:40.531059 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.531018 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d989c3fd-d744-4f23-8ef9-55a2347f735b-tls-cert\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.531059 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.531060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d989c3fd-d744-4f23-8ef9-55a2347f735b-oidc-ca\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.531320 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.531156 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdkgm\" (UniqueName: \"kubernetes.io/projected/d989c3fd-d744-4f23-8ef9-55a2347f735b-kube-api-access-kdkgm\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.631988 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.631952 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdkgm\" (UniqueName: \"kubernetes.io/projected/d989c3fd-d744-4f23-8ef9-55a2347f735b-kube-api-access-kdkgm\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.632177 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.632037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d989c3fd-d744-4f23-8ef9-55a2347f735b-tls-cert\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.632177 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.632081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d989c3fd-d744-4f23-8ef9-55a2347f735b-oidc-ca\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.632768 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.632746 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/d989c3fd-d744-4f23-8ef9-55a2347f735b-oidc-ca\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.634551 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.634530 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d989c3fd-d744-4f23-8ef9-55a2347f735b-tls-cert\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.640529 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.640507 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdkgm\" (UniqueName: \"kubernetes.io/projected/d989c3fd-d744-4f23-8ef9-55a2347f735b-kube-api-access-kdkgm\") pod \"authorino-6b5b4757b4-42sb5\" (UID: \"d989c3fd-d744-4f23-8ef9-55a2347f735b\") " pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.744029 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.743988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b5b4757b4-42sb5" Jun 11 17:03:40.883227 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:40.883159 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b5b4757b4-42sb5"] Jun 11 17:03:40.886453 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:03:40.886427 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd989c3fd_d744_4f23_8ef9_55a2347f735b.slice/crio-3ac86e8cf273f5e8da6e277dd4b2f90127fbb98e13d40fb1f697cfe79c2a8cbb WatchSource:0}: Error finding container 3ac86e8cf273f5e8da6e277dd4b2f90127fbb98e13d40fb1f697cfe79c2a8cbb: Status 404 returned error can't find the container with id 3ac86e8cf273f5e8da6e277dd4b2f90127fbb98e13d40fb1f697cfe79c2a8cbb Jun 11 17:03:41.837552 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:41.837497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5b4757b4-42sb5" event={"ID":"d989c3fd-d744-4f23-8ef9-55a2347f735b","Type":"ContainerStarted","Data":"58065712d5c4760c30d96f8f96d1f6ccacd7e6bad3d7f16849c0037859c13eea"} Jun 11 17:03:41.837552 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:41.837554 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b5b4757b4-42sb5" event={"ID":"d989c3fd-d744-4f23-8ef9-55a2347f735b","Type":"ContainerStarted","Data":"3ac86e8cf273f5e8da6e277dd4b2f90127fbb98e13d40fb1f697cfe79c2a8cbb"} Jun 11 17:03:41.862451 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:41.862382 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b5b4757b4-42sb5" podStartSLOduration=1.523076037 podStartE2EDuration="1.862361125s" totalImagesPullingTime="339.285088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:03:41.860816837 +0000 UTC m=+1177.749948215" watchObservedRunningTime="2026-06-11 17:03:41.862361125 +0000 UTC m=+1177.751492503" Jun 11 17:03:41.895984 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:41.895932 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:03:41.896471 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:41.896412 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5676d68769-r94jq" podUID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" containerName="authorino" containerID="cri-o://ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc" gracePeriod=30 Jun 11 17:03:42.175442 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.175183 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:03:42.248686 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.248654 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert\") pod \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " Jun 11 17:03:42.248858 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.248735 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll4g2\" (UniqueName: \"kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2\") pod \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " Jun 11 17:03:42.248858 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.248816 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca\") pod \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\" (UID: \"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2\") " Jun 11 17:03:42.250934 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.250906 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2" (OuterVolumeSpecName: "kube-api-access-ll4g2") pod "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" (UID: "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2"). InnerVolumeSpecName "kube-api-access-ll4g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 17:03:42.254913 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.254850 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" (UID: "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 17:03:42.261142 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.261119 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" (UID: "9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 17:03:42.349613 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.349556 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll4g2\" (UniqueName: \"kubernetes.io/projected/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-kube-api-access-ll4g2\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:03:42.349773 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.349617 2557 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-oidc-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:03:42.349773 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.349634 2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2-tls-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:03:42.843241 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.843145 2557 generic.go:356] "Generic (PLEG): container finished" podID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" containerID="ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc" exitCode=0 Jun 11 17:03:42.843241 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.843189 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5676d68769-r94jq" Jun 11 17:03:42.843241 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.843230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5676d68769-r94jq" event={"ID":"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2","Type":"ContainerDied","Data":"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc"} Jun 11 17:03:42.843805 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.843272 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5676d68769-r94jq" event={"ID":"9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2","Type":"ContainerDied","Data":"c6837f60ae18049244371930f67325976af7780b8fb2f6198e007dec6322d997"} Jun 11 17:03:42.843805 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.843294 2557 scope.go:117] "RemoveContainer" containerID="ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc" Jun 11 17:03:42.852504 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.852486 2557 scope.go:117] "RemoveContainer" containerID="ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc" Jun 11 17:03:42.852775 ip-10-0-128-197 kubenswrapper[2557]: E0611 17:03:42.852758 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc\": container with ID starting with ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc not found: ID does not exist" containerID="ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc" Jun 11 17:03:42.852847 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.852785 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc"} err="failed to get container status \"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc\": rpc error: code = NotFound desc = could not find container \"ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc\": container with ID starting with ae0ab68b55ef277add519ea6711cd3b91e6e902e8c2fa9862bf07b6d77b088cc not found: ID does not exist" Jun 11 17:03:42.862564 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.862540 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:03:42.866657 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:42.866638 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5676d68769-r94jq"] Jun 11 17:03:44.518203 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:03:44.518174 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" path="/var/lib/kubelet/pods/9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2/volumes" Jun 11 17:04:04.641233 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:04:04.641110 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:04:04.673421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:04:04.648328 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:05:10.491677 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.491638 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c786bc658-2xq68"] Jun 11 17:05:10.492149 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.492046 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" containerName="authorino" Jun 11 17:05:10.492149 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.492060 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" containerName="authorino" Jun 11 17:05:10.492149 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.492149 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9770912d-4f93-4e9a-8e4a-e8d5b9f8e5c2" containerName="authorino" Jun 11 17:05:10.494250 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.494234 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.498192 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.498171 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 17:05:10.498479 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.498456 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pzdvc\"" Jun 11 17:05:10.498646 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.498632 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 17:05:10.510390 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.510364 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c786bc658-2xq68"] Jun 11 17:05:10.564890 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.564851 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x22h\" (UniqueName: \"kubernetes.io/projected/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-kube-api-access-7x22h\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.565068 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.564963 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-webhook-cert\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.666374 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.666338 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-webhook-cert\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.666548 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.666402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x22h\" (UniqueName: \"kubernetes.io/projected/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-kube-api-access-7x22h\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.668888 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.668867 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-webhook-cert\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.677360 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.677332 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x22h\" (UniqueName: \"kubernetes.io/projected/a8c1fad9-8964-4814-84be-8d2f44b9e9ce-kube-api-access-7x22h\") pod \"maas-controller-7c786bc658-2xq68\" (UID: \"a8c1fad9-8964-4814-84be-8d2f44b9e9ce\") " pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.805082 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.804995 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:10.941162 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:10.941130 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c786bc658-2xq68"] Jun 11 17:05:10.942618 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:05:10.942564 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c1fad9_8964_4814_84be_8d2f44b9e9ce.slice/crio-c6ebf1aa224cfa62431f70ad422c6466bae884bd19e2f73148ffa372b7582678 WatchSource:0}: Error finding container c6ebf1aa224cfa62431f70ad422c6466bae884bd19e2f73148ffa372b7582678: Status 404 returned error can't find the container with id c6ebf1aa224cfa62431f70ad422c6466bae884bd19e2f73148ffa372b7582678 Jun 11 17:05:11.279912 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:11.279877 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c786bc658-2xq68" event={"ID":"a8c1fad9-8964-4814-84be-8d2f44b9e9ce","Type":"ContainerStarted","Data":"c6ebf1aa224cfa62431f70ad422c6466bae884bd19e2f73148ffa372b7582678"} Jun 11 17:05:12.286483 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:12.286447 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c786bc658-2xq68" event={"ID":"a8c1fad9-8964-4814-84be-8d2f44b9e9ce","Type":"ContainerStarted","Data":"909c72f62fd0a7f1bb7204356e7c03fc904649655bc3b7cac4af1b7a49c716d0"} Jun 11 17:05:12.286954 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:12.286573 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:05:12.305345 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:12.305288 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c786bc658-2xq68" podStartSLOduration=1.8441150290000001 podStartE2EDuration="2.305270587s" totalImagesPullingTime="461.155558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:05:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:05:12.304087707 +0000 UTC m=+1268.193219084" watchObservedRunningTime="2026-06-11 17:05:12.305270587 +0000 UTC m=+1268.194401964" Jun 11 17:05:23.298109 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:05:23.298075 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c786bc658-2xq68" Jun 11 17:09:04.674029 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:09:04.673915 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:09:04.681594 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:09:04.681554 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:14:04.708409 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:14:04.708306 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:14:04.715109 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:14:04.715087 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:15:24.940356 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:24.940273 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 17:15:24.940837 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:24.940477 2557 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" podUID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" containerName="manager" containerID="cri-o://a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c" gracePeriod=10 Jun 11 17:15:25.199510 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.199450 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 17:15:25.284187 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.284156 2557 generic.go:356] "Generic (PLEG): container finished" podID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" containerID="a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c" exitCode=0 Jun 11 17:15:25.284352 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.284199 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" Jun 11 17:15:25.284352 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.284227 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" event={"ID":"20a8d8a5-cda8-440c-b039-fa3aab9c9627","Type":"ContainerDied","Data":"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c"} Jun 11 17:15:25.284352 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.284280 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t" event={"ID":"20a8d8a5-cda8-440c-b039-fa3aab9c9627","Type":"ContainerDied","Data":"5ba20ed6d9d04bf8f02f06ec836eadc3432610f1d04ee2acc1ee3e5859c409fd"} Jun 11 17:15:25.284352 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.284302 2557 scope.go:117] "RemoveContainer" containerID="a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c" Jun 11 17:15:25.293189 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.293168 2557 scope.go:117] "RemoveContainer" containerID="a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c" Jun 11 17:15:25.293469 ip-10-0-128-197 kubenswrapper[2557]: E0611 17:15:25.293435 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c\": container with ID starting with a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c not found: ID does not exist" containerID="a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c" Jun 11 17:15:25.293542 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.293478 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c"} err="failed to get container status \"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c\": rpc error: code = NotFound desc = could not find container \"a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c\": container with ID starting with a1703b5d812b069d07016188823c4e375acd9da0b21f7f8cb437343a65f94d5c not found: ID does not exist" Jun 11 17:15:25.314878 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.314850 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume\") pod \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " Jun 11 17:15:25.314976 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.314904 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj749\" (UniqueName: \"kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749\") pod \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\" (UID: \"20a8d8a5-cda8-440c-b039-fa3aab9c9627\") " Jun 11 17:15:25.315301 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.315272 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "20a8d8a5-cda8-440c-b039-fa3aab9c9627" (UID: "20a8d8a5-cda8-440c-b039-fa3aab9c9627"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 17:15:25.316951 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.316922 2557 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749" (OuterVolumeSpecName: "kube-api-access-bj749") pod "20a8d8a5-cda8-440c-b039-fa3aab9c9627" (UID: "20a8d8a5-cda8-440c-b039-fa3aab9c9627"). InnerVolumeSpecName "kube-api-access-bj749". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 17:15:25.415749 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.415713 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20a8d8a5-cda8-440c-b039-fa3aab9c9627-extensions-socket-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:15:25.415749 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.415743 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj749\" (UniqueName: \"kubernetes.io/projected/20a8d8a5-cda8-440c-b039-fa3aab9c9627-kube-api-access-bj749\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\"" Jun 11 17:15:25.609979 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.609939 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 17:15:25.612272 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:25.612248 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-57k6t"] Jun 11 17:15:26.517854 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:15:26.517823 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" path="/var/lib/kubelet/pods/20a8d8a5-cda8-440c-b039-fa3aab9c9627/volumes" Jun 11 17:16:31.225436 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.225403 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz"] Jun 11 17:16:31.225897 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.225841 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" containerName="manager" Jun 11 17:16:31.225897 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.225854 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" containerName="manager" Jun 11 17:16:31.225971 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.225923 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a8d8a5-cda8-440c-b039-fa3aab9c9627" containerName="manager" Jun 11 17:16:31.229018 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.229002 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.232221 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.232205 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l78nt\"" Jun 11 17:16:31.233888 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.233868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7qq\" (UniqueName: \"kubernetes.io/projected/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-kube-api-access-kp7qq\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.233943 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.233907 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.245981 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.245955 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz"] Jun 11 17:16:31.334628 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.334562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7qq\" (UniqueName: \"kubernetes.io/projected/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-kube-api-access-kp7qq\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.334820 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.334646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.335033 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.335014 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.348954 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.348901 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7qq\" (UniqueName: \"kubernetes.io/projected/6a57f070-2dfa-4fe9-8fa6-cbd4e6072338-kube-api-access-kp7qq\") pod \"kuadrant-operator-controller-manager-86878c769f-7f8tz\" (UID: \"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.539777 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.539675 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:31.696613 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:31.696571 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz"] Jun 11 17:16:31.698878 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:16:31.698846 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a57f070_2dfa_4fe9_8fa6_cbd4e6072338.slice/crio-4e722ddba3400573afdf0fd602146ba85aaeb3da55bcd08db7b6397a53f39659 WatchSource:0}: Error finding container 4e722ddba3400573afdf0fd602146ba85aaeb3da55bcd08db7b6397a53f39659: Status 404 returned error can't find the container with id 4e722ddba3400573afdf0fd602146ba85aaeb3da55bcd08db7b6397a53f39659 Jun 11 17:16:32.612299 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:32.612266 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" event={"ID":"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338","Type":"ContainerStarted","Data":"362e9e8390e2da7be6569948adfe7be2f901bfb2d177c6eb3d7221f5b9deabb1"} Jun 11 17:16:32.612731 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:32.612313 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" event={"ID":"6a57f070-2dfa-4fe9-8fa6-cbd4e6072338","Type":"ContainerStarted","Data":"4e722ddba3400573afdf0fd602146ba85aaeb3da55bcd08db7b6397a53f39659"} Jun 11 17:16:32.612731 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:32.612425 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:16:32.653045 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:32.652998 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" podStartSLOduration=1.652979743 podStartE2EDuration="1.652979743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 17:16:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:16:32.651207564 +0000 UTC m=+1948.540338978" watchObservedRunningTime="2026-06-11 17:16:32.652979743 +0000 UTC m=+1948.542111121" Jun 11 17:16:43.621854 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:16:43.621823 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7f8tz" Jun 11 17:19:04.744341 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:19:04.744313 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:19:04.751175 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:19:04.751154 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:21:54.293760 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:21:54.293644 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7c786bc658-2xq68" podUID="a8c1fad9-8964-4814-84be-8d2f44b9e9ce" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.47:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jun 11 17:22:04.294329 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:22:04.294288 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7c786bc658-2xq68" podUID="a8c1fad9-8964-4814-84be-8d2f44b9e9ce" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.47:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jun 11 17:24:04.775858 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:24:04.775756 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:24:04.790871 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:24:04.790830 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s8cmj_4c925d43-c0ac-4c8c-99a2-fd146d4b9644/ovn-acl-logging/0.log" Jun 11 17:26:48.397175 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:48.397136 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b5b4757b4-42sb5_d989c3fd-d744-4f23-8ef9-55a2347f735b/authorino/0.log" Jun 11 17:26:53.805840 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:53.805815 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5bc764bbcb-9mrjr_4433523c-0cf8-46ca-ba3b-046e53deab54/maas-api/0.log" Jun 11 17:26:54.160538 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:54.160461 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c786bc658-2xq68_a8c1fad9-8964-4814-84be-8d2f44b9e9ce/manager/0.log" Jun 11 17:26:54.772144 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:54.772118 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f54898884-z6trv_404ac9b8-25ce-41b4-9700-196fe5af3aca/copy-manifests/0.log" Jun 11 17:26:54.785528 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:54.785504 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f54898884-z6trv_404ac9b8-25ce-41b4-9700-196fe5af3aca/manager/0.log" Jun 11 17:26:56.196169 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:56.196136 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b5b4757b4-42sb5_d989c3fd-d744-4f23-8ef9-55a2347f735b/authorino/0.log" Jun 11 17:26:56.560451 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:56.560419 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jshq4_cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd/kuadrant-console-plugin/0.log" Jun 11 17:26:56.806742 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:56.806704 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7f8tz_6a57f070-2dfa-4fe9-8fa6-cbd4e6072338/manager/0.log" Jun 11 17:26:57.646421 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:57.646385 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-588969d7f4-5vlxw_8fa49174-5b8a-4fa9-ace4-71a8feefd759/kube-auth-proxy/0.log" Jun 11 17:26:58.010874 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:58.010841 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-h64r7_1536f481-6401-4ea1-b14a-b52fee53af2a/payload-pre-processing/0.log" Jun 11 17:26:58.251307 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:58.251281 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log" Jun 11 17:26:59.092321 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:59.092289 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x_8f6a726e-ecab-4bed-ae85-6e4395d588c1/storage-initializer/0.log" Jun 11 17:26:59.100510 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:26:59.100486 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-b7x2x_8f6a726e-ecab-4bed-ae85-6e4395d588c1/main/0.log" Jun 11 17:27:07.292696 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:07.292657 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7sddr_1219414a-d398-4f74-a6f2-b59466fbf0da/global-pull-secret-syncer/0.log" Jun 11 17:27:07.493801 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:07.493768 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zg2zm_7a8fc95c-aaf4-4f9e-bcc7-d6bdb8ae5e88/konnectivity-agent/0.log" Jun 11 17:27:07.524889 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:07.524861 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-197.ec2.internal_0ffe0ae6d94a8ca6aa9430e1dfacd020/haproxy/0.log" Jun 11 17:27:12.372639 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:12.372605 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b5b4757b4-42sb5_d989c3fd-d744-4f23-8ef9-55a2347f735b/authorino/0.log" Jun 11 17:27:12.497677 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:12.497649 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jshq4_cbcb9ec6-8798-4004-bd68-49f8c1dbe0fd/kuadrant-console-plugin/0.log" Jun 11 17:27:12.619883 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:12.619845 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7f8tz_6a57f070-2dfa-4fe9-8fa6-cbd4e6072338/manager/0.log" Jun 11 17:27:14.286550 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.286519 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/alertmanager/0.log" Jun 11 17:27:14.320432 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.320407 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/config-reloader/0.log" Jun 11 17:27:14.341137 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.341110 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/kube-rbac-proxy-web/0.log" Jun 11 17:27:14.370474 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.370450 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/kube-rbac-proxy/0.log" Jun 11 17:27:14.394242 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.394211 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/kube-rbac-proxy-metric/0.log" Jun 11 17:27:14.417140 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.417118 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/prom-label-proxy/0.log" Jun 11 17:27:14.444684 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.444639 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_20b03e5d-4c8a-4db1-aa34-044945716ac5/init-config-reloader/0.log" Jun 11 17:27:14.601979 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.601906 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b6c985589-kghpg_4dd90dad-f793-4a32-8011-a963805fd8b6/metrics-server/0.log" Jun 11 17:27:14.628890 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.628867 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-f57fv_c81680e2-bdc4-4ffe-8f52-3b4047dcc88e/monitoring-plugin/0.log" Jun 11 17:27:14.912635 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.912604 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcvws_7f7f016e-b12b-49b2-bb29-ef71237598ab/node-exporter/0.log" Jun 11 17:27:14.939764 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.939742 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcvws_7f7f016e-b12b-49b2-bb29-ef71237598ab/kube-rbac-proxy/0.log" Jun 11 17:27:14.968739 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:14.968716 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jcvws_7f7f016e-b12b-49b2-bb29-ef71237598ab/init-textfile/0.log" Jun 11 17:27:15.105758 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.105719 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/prometheus/0.log" Jun 11 17:27:15.127149 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.127083 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/config-reloader/0.log" Jun 11 17:27:15.151882 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.151860 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/thanos-sidecar/0.log" Jun 11 17:27:15.208136 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.208107 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/kube-rbac-proxy-web/0.log" Jun 11 17:27:15.252447 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.252417 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/kube-rbac-proxy/0.log" Jun 11 17:27:15.310509 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.310483 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/kube-rbac-proxy-thanos/0.log" Jun 11 17:27:15.369999 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.369974 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_002d4b8b-0c30-4de2-bf55-dd0091eac874/init-config-reloader/0.log" Jun 11 17:27:15.414044 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.413974 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-n5spj_5ccc1a0d-1568-488f-94a2-cef16964a7d9/prometheus-operator/0.log" Jun 11 17:27:15.439456 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.439434 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-n5spj_5ccc1a0d-1568-488f-94a2-cef16964a7d9/kube-rbac-proxy/0.log" Jun 11 17:27:15.516532 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.516498 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fc455c9bf-thmmj_cdfa4b33-c4f3-4fad-9db5-26939cb0a094/telemeter-client/0.log" Jun 11 17:27:15.544067 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.544046 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fc455c9bf-thmmj_cdfa4b33-c4f3-4fad-9db5-26939cb0a094/reload/0.log" Jun 11 17:27:15.569836 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:15.569808 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-fc455c9bf-thmmj_cdfa4b33-c4f3-4fad-9db5-26939cb0a094/kube-rbac-proxy/0.log" Jun 11 17:27:16.965228 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:16.965198 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-6pw9w_55fa74ba-8ddc-4f2e-82f1-2ef5e3238a8e/networking-console-plugin/0.log" Jun 11 17:27:18.506744 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.506711 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn"] Jun 11 17:27:18.510374 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.510356 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.512900 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.512876 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7wbff\"/\"kube-root-ca.crt\"" Jun 11 17:27:18.513703 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.513687 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7wbff\"/\"openshift-service-ca.crt\"" Jun 11 17:27:18.513776 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.513724 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7wbff\"/\"default-dockercfg-wqlrp\"" Jun 11 17:27:18.520770 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.520739 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn"] Jun 11 17:27:18.574144 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.574105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-podres\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.574316 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.574230 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-sys\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.574316 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.574292 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-lib-modules\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.574316 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.574311 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn4fn\" (UniqueName: \"kubernetes.io/projected/702ba048-38a2-40f6-ac74-fd972aa88b00-kube-api-access-tn4fn\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.574431 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.574329 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-proc\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675359 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-lib-modules\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675359 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675358 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn4fn\" (UniqueName: \"kubernetes.io/projected/702ba048-38a2-40f6-ac74-fd972aa88b00-kube-api-access-tn4fn\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-proc\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675412 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-podres\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-sys\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675499 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-lib-modules\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675524 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-proc\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675555 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-sys\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.675638 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.675626 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/702ba048-38a2-40f6-ac74-fd972aa88b00-podres\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.685096 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.685065 2557 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn4fn\" (UniqueName: \"kubernetes.io/projected/702ba048-38a2-40f6-ac74-fd972aa88b00-kube-api-access-tn4fn\") pod \"perf-node-gather-daemonset-8zwpn\" (UID: \"702ba048-38a2-40f6-ac74-fd972aa88b00\") " pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.822890 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.822786 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:18.966830 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:18.966803 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn"] Jun 11 17:27:18.968470 ip-10-0-128-197 kubenswrapper[2557]: W0611 17:27:18.968435 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod702ba048_38a2_40f6_ac74_fd972aa88b00.slice/crio-80b8f3de075486e02c1e5430a510c179bd6aedb475aceea84b590d32db69db7a WatchSource:0}: Error finding container 80b8f3de075486e02c1e5430a510c179bd6aedb475aceea84b590d32db69db7a: Status 404 returned error can't find the container with id 80b8f3de075486e02c1e5430a510c179bd6aedb475aceea84b590d32db69db7a Jun 11 17:27:19.123976 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:19.123900 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-6gm99_e828a0bd-57dc-4224-a2c4-2ac58e5c09d3/volume-data-source-validator/0.log" Jun 11 17:27:19.801542 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:19.801504 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" event={"ID":"702ba048-38a2-40f6-ac74-fd972aa88b00","Type":"ContainerStarted","Data":"e81a0ac21edb23b7265d17ea32808b84f1149db9263fa2608922f86014f21d9c"} Jun 11 17:27:19.801971 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:19.801567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" event={"ID":"702ba048-38a2-40f6-ac74-fd972aa88b00","Type":"ContainerStarted","Data":"80b8f3de075486e02c1e5430a510c179bd6aedb475aceea84b590d32db69db7a"} Jun 11 17:27:19.801971 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:19.801631 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:19.823424 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:19.823375 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" podStartSLOduration=1.8233609990000001 podStartE2EDuration="1.823360999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 17:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 17:27:19.820642528 +0000 UTC m=+2595.709773904" watchObservedRunningTime="2026-06-11 17:27:19.823360999 +0000 UTC m=+2595.712492377" Jun 11 17:27:20.249657 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:20.249633 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2dcgz_70b96e71-be86-45c3-9d86-4f7ec5c52525/dns/0.log" Jun 11 17:27:20.277728 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:20.277701 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2dcgz_70b96e71-be86-45c3-9d86-4f7ec5c52525/kube-rbac-proxy/0.log" Jun 11 17:27:20.461690 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:20.461654 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k2nzc_4091c34c-6597-4aef-9417-1aa198593e6f/dns-node-resolver/0.log" Jun 11 17:27:21.130150 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:21.130126 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bjlhc_d6678642-6849-4cda-9772-fcf28a9eb640/node-ca/0.log" Jun 11 17:27:22.320513 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:22.320483 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-588969d7f4-5vlxw_8fa49174-5b8a-4fa9-ace4-71a8feefd759/kube-auth-proxy/0.log" Jun 11 17:27:22.476307 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:22.476278 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-h64r7_1536f481-6401-4ea1-b14a-b52fee53af2a/payload-pre-processing/0.log" Jun 11 17:27:22.562423 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:22.562395 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log" Jun 11 17:27:23.313721 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:23.313693 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gwwn2_58ac9bf0-ee3c-4e93-a17d-b78f099d991f/serve-healthcheck-canary/0.log" Jun 11 17:27:24.172665 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:24.172637 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ksrjf_a426988b-e088-4524-8df0-21f725d06d4e/kube-rbac-proxy/0.log" Jun 11 17:27:24.207714 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:24.207687 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ksrjf_a426988b-e088-4524-8df0-21f725d06d4e/exporter/0.log" Jun 11 17:27:24.250382 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:24.250355 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ksrjf_a426988b-e088-4524-8df0-21f725d06d4e/extractor/0.log" Jun 11 17:27:25.816803 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:25.816769 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7wbff/perf-node-gather-daemonset-8zwpn" Jun 11 17:27:26.867634 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:26.867602 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5bc764bbcb-9mrjr_4433523c-0cf8-46ca-ba3b-046e53deab54/maas-api/0.log" Jun 11 17:27:27.003935 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:27.003903 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c786bc658-2xq68_a8c1fad9-8964-4814-84be-8d2f44b9e9ce/manager/0.log" Jun 11 17:27:27.383029 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:27.383002 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f54898884-z6trv_404ac9b8-25ce-41b4-9700-196fe5af3aca/manager/0.log" Jun 11 17:27:27.409747 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:27.409722 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f54898884-z6trv_404ac9b8-25ce-41b4-9700-196fe5af3aca/copy-manifests/0.log" Jun 11 17:27:29.231196 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:29.231165 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57b9974d87-4p7sj_a85ac72d-d3e6-4de9-a53f-5f1c58f10786/manager/0.log" Jun 11 17:27:31.880866 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:31.880838 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-588969d7f4-5vlxw_8fa49174-5b8a-4fa9-ace4-71a8feefd759/kube-auth-proxy/0.log" Jun 11 17:27:32.018660 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:32.018630 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-h64r7_1536f481-6401-4ea1-b14a-b52fee53af2a/payload-pre-processing/0.log" Jun 11 17:27:32.087506 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:32.087480 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log" Jun 11 17:27:33.253137 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:33.253107 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-588969d7f4-5vlxw_8fa49174-5b8a-4fa9-ace4-71a8feefd759/kube-auth-proxy/0.log" Jun 11 17:27:33.385312 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:33.385283 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-h64r7_1536f481-6401-4ea1-b14a-b52fee53af2a/payload-pre-processing/0.log" Jun 11 17:27:33.452980 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:33.452944 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log" Jun 11 17:27:34.592565 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:34.592532 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-6r672_eb7b202c-5547-4548-91e8-c3c9679f12cc/migrator/0.log" Jun 11 17:27:34.614155 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:34.614127 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-6r672_eb7b202c-5547-4548-91e8-c3c9679f12cc/graceful-termination/0.log" Jun 11 17:27:35.017711 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:35.017676 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-r8gzr_6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11/kube-storage-version-migrator-operator/1.log" Jun 11 17:27:35.018430 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:35.018410 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-r8gzr_6b70d9e6-fc73-4f87-bb35-b79b2ab0ff11/kube-storage-version-migrator-operator/0.log" Jun 11 17:27:35.203705 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:35.203677 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-588969d7f4-5vlxw_8fa49174-5b8a-4fa9-ace4-71a8feefd759/kube-auth-proxy/0.log" Jun 11 17:27:35.330829 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:35.330747 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-h64r7_1536f481-6401-4ea1-b14a-b52fee53af2a/payload-pre-processing/0.log" Jun 11 17:27:35.396356 ip-10-0-128-197 kubenswrapper[2557]: I0611 17:27:35.396329 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ddbb9c9cc-g52gs_7b91dede-ffde-4c85-a5a7-382cc937f42a/router/0.log"