Jun 08 16:30:43.902895 ip-10-0-141-92 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 08 16:30:43.902910 ip-10-0-141-92 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 08 16:30:43.902920 ip-10-0-141-92 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 08 16:30:43.903253 ip-10-0-141-92 systemd[1]: Failed to start Kubernetes Kubelet. Jun 08 16:30:53.576074 ip-10-0-141-92 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 08 16:30:53.576088 ip-10-0-141-92 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 11ee36ab3eee4a73942cee8286992e20 -- Jun 08 16:33:15.476013 ip-10-0-141-92 systemd[1]: Starting Kubernetes Kubelet... Jun 08 16:33:15.932761 ip-10-0-141-92 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 16:33:15.932761 ip-10-0-141-92 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 16:33:15.932761 ip-10-0-141-92 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 16:33:15.932761 ip-10-0-141-92 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 16:33:15.932761 ip-10-0-141-92 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 16:33:15.935785 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.935691 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 16:33:15.941670 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941620 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941672 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941678 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941682 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941685 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941688 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941691 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941694 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941697 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941699 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941702 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941705 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941708 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941710 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941713 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941716 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941718 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941721 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941723 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.941724 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941726 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941731 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941735 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941738 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941741 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941744 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941747 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941751 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941754 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941758 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941762 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941765 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941768 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941771 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941773 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941776 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941778 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941781 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941783 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.942209 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941786 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941788 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941791 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941793 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941796 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941798 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941801 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941804 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941806 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941809 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941812 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941814 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941817 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941821 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941825 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941828 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941832 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941834 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941837 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.942684 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941839 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941842 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941844 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941847 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941850 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941852 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941855 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941858 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941860 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941863 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941866 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941868 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941871 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941873 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941876 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941879 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941881 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941884 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941886 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941889 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.943137 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941891 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941895 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941897 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941900 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941903 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941906 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941908 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941913 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941916 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941919 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941921 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941924 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941926 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941929 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941932 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941934 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941937 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941940 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941942 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941945 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.943606 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941948 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.941950 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943548 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943556 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943560 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943563 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943566 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943569 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943571 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943574 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943577 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943579 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943582 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943584 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943588 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943590 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943593 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943596 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943598 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943601 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.944094 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943605 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943607 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943610 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943613 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943617 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943620 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943622 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943625 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943628 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943648 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943652 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943657 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943661 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943665 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943668 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943670 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943673 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943676 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943679 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943681 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.944627 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943684 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943686 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943690 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943694 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943697 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943699 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943703 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943706 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943708 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943711 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943715 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943720 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943723 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943726 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943730 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943732 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943735 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943738 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.945120 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943743 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943747 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943750 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943753 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943756 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943758 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943761 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943763 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943766 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943769 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943772 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943774 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943777 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943781 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943783 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943786 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943788 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943791 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943794 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943797 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.945750 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943799 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943802 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943805 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943808 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943811 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943813 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943816 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943818 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943820 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943823 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943826 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943828 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943831 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943834 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943836 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943839 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943842 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943844 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943847 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943850 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.946418 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943852 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943855 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.943858 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944259 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944264 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944267 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944270 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944273 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944276 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944279 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944282 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944284 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944287 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944290 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944293 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944295 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944298 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944301 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944304 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944307 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.947013 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944309 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944312 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944314 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944316 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944319 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944321 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944324 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944327 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944329 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944332 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944334 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944337 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944339 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944344 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944348 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944351 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944354 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944356 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944358 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944361 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.947508 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944364 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944366 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944369 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944371 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944374 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944376 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944379 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944381 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944383 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944386 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944388 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944390 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944394 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944398 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944400 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944403 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944406 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944409 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944411 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.947999 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944414 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944417 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944419 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944422 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944424 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944427 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944429 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944432 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944434 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944437 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944439 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944442 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944444 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944447 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944450 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944453 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944456 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944459 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944461 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.948503 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944464 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944466 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944469 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944471 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944474 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944476 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944479 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944481 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944483 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944486 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944488 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944491 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944493 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944496 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944498 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944501 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944503 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944506 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944508 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944511 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.948979 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944514 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944517 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944521 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.944524 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944609 2559 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944620 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944644 2559 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944653 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944658 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944661 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944665 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944670 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944673 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944676 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944679 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944682 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944685 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944688 2559 flags.go:64] FLAG: --cgroup-root="" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944691 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944694 2559 flags.go:64] FLAG: --client-ca-file="" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944697 2559 flags.go:64] FLAG: --cloud-provider="external" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944700 2559 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944705 2559 flags.go:64] FLAG: --cluster-domain="" Jun 08 16:33:15.949464 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944708 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944711 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944714 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944717 2559 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944721 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944724 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944727 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944731 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944734 2559 flags.go:64] FLAG: --contention-profiling="false" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944737 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944740 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944744 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944747 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944751 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944754 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944757 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944760 2559 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944763 2559 flags.go:64] FLAG: --enable-server="true" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944766 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944770 2559 flags.go:64] FLAG: --event-burst="100" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944774 2559 flags.go:64] FLAG: --event-qps="50" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944777 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944780 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944783 2559 flags.go:64] FLAG: --eviction-hard="" Jun 08 16:33:15.950044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944787 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944790 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944793 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944796 2559 flags.go:64] FLAG: --eviction-soft="" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944798 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944801 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944804 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944807 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944809 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944812 2559 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944817 2559 flags.go:64] FLAG: --feature-gates="" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944821 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944824 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944827 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944830 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944837 2559 flags.go:64] FLAG: --healthz-port="10248" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944839 2559 flags.go:64] FLAG: --help="false" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944842 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-141-92.ec2.internal" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944846 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944849 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944853 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944856 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944859 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944862 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 16:33:15.950612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944865 2559 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944868 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944870 2559 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944874 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944877 2559 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944879 2559 flags.go:64] FLAG: --kube-reserved="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944882 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944885 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944889 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944892 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944895 2559 flags.go:64] FLAG: --lock-file="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944898 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944901 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944904 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944910 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944913 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944915 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944918 2559 flags.go:64] FLAG: --logging-format="text" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944922 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944926 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944928 2559 flags.go:64] FLAG: --manifest-url="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944931 2559 flags.go:64] FLAG: --manifest-url-header="" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944936 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944940 2559 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944945 2559 flags.go:64] FLAG: --max-pods="110" Jun 08 16:33:15.951239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944947 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944951 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944953 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944956 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944960 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944963 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944966 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944974 2559 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944977 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944980 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944983 2559 flags.go:64] FLAG: --pod-cidr="" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944986 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944992 2559 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944994 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.944997 2559 flags.go:64] FLAG: --pods-per-core="0" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945000 2559 flags.go:64] FLAG: --port="10250" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945003 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945006 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0395b395baf375e60" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945010 2559 flags.go:64] FLAG: --qos-reserved="" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945012 2559 flags.go:64] FLAG: --read-only-port="10255" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945015 2559 flags.go:64] FLAG: --register-node="true" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945018 2559 flags.go:64] FLAG: --register-with-taints="" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945022 2559 flags.go:64] FLAG: --registry-burst="10" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945025 2559 flags.go:64] FLAG: --registry-qps="5" Jun 08 16:33:15.952001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945027 2559 flags.go:64] FLAG: --reserved-cpus="" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945032 2559 flags.go:64] FLAG: --reserved-memory="" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945036 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945039 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945042 2559 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945044 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945047 2559 flags.go:64] FLAG: --runonce="false" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945051 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945054 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945057 2559 flags.go:64] FLAG: --seccomp-default="false" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945060 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945063 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945066 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945070 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945073 2559 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945076 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945079 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945081 2559 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945084 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945087 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945090 2559 flags.go:64] FLAG: --system-cgroups="" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945094 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945099 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945102 2559 flags.go:64] FLAG: --tls-cert-file="" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945104 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 16:33:15.952585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945109 2559 flags.go:64] FLAG: --tls-min-version="" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945112 2559 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945115 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945118 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945121 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945124 2559 flags.go:64] FLAG: --v="2" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945128 2559 flags.go:64] FLAG: --version="false" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945132 2559 flags.go:64] FLAG: --vmodule="" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945138 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.945141 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945772 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945787 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945792 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945797 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945803 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945807 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945812 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945816 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945820 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945824 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945828 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945834 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.953220 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945844 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945849 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945853 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945857 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945862 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945866 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945870 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945875 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945879 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945883 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945887 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945892 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945900 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945905 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945909 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945913 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945918 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945923 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945929 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945933 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.953772 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945937 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945942 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945946 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945950 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945956 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945964 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945969 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945974 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945981 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945986 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945991 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.945996 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946002 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946007 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946012 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946016 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946020 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946030 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946036 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.954248 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946041 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946045 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946049 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946054 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946058 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946062 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946066 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946070 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946075 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946079 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946084 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946095 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946099 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946103 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946108 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946112 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946116 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946122 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946126 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.954734 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946131 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946137 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946143 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946148 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946157 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946162 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946166 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946171 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946176 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946181 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946185 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946189 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946194 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946198 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946202 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946206 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946211 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946220 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946224 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946228 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.955193 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946232 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946237 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946242 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946246 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946253 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946718 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946851 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946857 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.946861 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.947619 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.954926 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 16:33:15.955696 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.954946 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.955829 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.955838 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955900 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955905 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955909 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955912 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955915 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955917 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955920 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955923 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955926 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955929 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955931 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955934 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955936 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955939 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955941 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955944 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955947 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.955975 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955950 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955954 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955958 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955962 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955965 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955968 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955971 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955973 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955976 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955979 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955982 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955985 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955988 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955991 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955994 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.955997 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956000 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956002 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956006 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.956454 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956009 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956012 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956015 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956018 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956021 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956024 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956027 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956029 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956032 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956035 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956037 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956040 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956043 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956045 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956048 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956051 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956054 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956056 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956059 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.956947 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956061 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956064 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956066 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956069 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956071 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956074 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956077 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956079 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956082 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956084 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956087 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956089 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956092 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956095 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956097 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956100 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956107 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956110 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956113 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956115 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.957430 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956118 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956120 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956123 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956125 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956128 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956131 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956133 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956136 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956139 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956142 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956144 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956147 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956149 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956152 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956155 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956157 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956160 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956162 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956166 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956170 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.957927 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956173 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956175 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956178 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956180 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.956186 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956306 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956310 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956313 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956316 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956319 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956322 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956325 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956328 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956330 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956333 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 16:33:15.958410 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956335 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956339 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956341 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956344 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956346 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956349 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956352 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956355 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956358 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956360 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956363 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956366 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956370 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956374 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956377 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956381 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956384 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956386 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 16:33:15.958798 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956389 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956391 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956394 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956397 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956399 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956402 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956404 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956407 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956409 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956412 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956414 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956417 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956419 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956422 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956424 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956427 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956430 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956433 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956436 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956438 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 16:33:15.959225 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956440 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956443 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956446 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956448 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956451 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956454 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956456 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956459 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956461 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956464 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956467 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956469 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956472 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956474 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956477 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956480 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956482 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956485 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956488 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956490 2559 feature_gate.go:328] unrecognized feature gate: Example Jun 08 16:33:15.959787 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956492 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956495 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956497 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956500 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956502 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956505 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956508 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956510 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956513 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956516 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956518 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956520 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956524 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956526 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956528 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956531 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956533 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956536 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956538 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956541 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 16:33:15.960267 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956543 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956546 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956548 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956551 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956553 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956556 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956558 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956561 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956563 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956566 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:15.956568 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.956573 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.956729 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 16:33:15.960760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.959087 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 16:33:15.961097 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.960884 2559 server.go:1019] "Starting client certificate rotation" Jun 08 16:33:15.961097 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.960988 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 16:33:15.961097 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.961025 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 16:33:15.963825 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.963809 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 16:33:15.965009 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.964991 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 16:33:15.975988 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.975967 2559 log.go:25] "Validated CRI v1 runtime API" Jun 08 16:33:15.980454 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.980432 2559 log.go:25] "Validated CRI v1 image API" Jun 08 16:33:15.981326 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.981293 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 16:33:15.987182 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.987154 2559 fs.go:135] Filesystem UUIDs: map[543b242b-f9b5-44fe-b9ae-ae059b394a2d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fc80b23d-8c61-4988-9284-279e458afcb4:/dev/nvme0n1p3] Jun 08 16:33:15.987297 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.987179 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 16:33:15.991853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.991820 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 16:33:15.993540 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.993403 2559 manager.go:216] Machine: {Timestamp:2026-06-08 16:33:15.992029502 +0000 UTC m=+0.378217888 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102020 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b35a1f40da114ee07307584c3d07d SystemUUID:ec2b35a1-f40d-a114-ee07-307584c3d07d BootID:11ee36ab-3eee-4a73-942c-ee8286992e20 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:f2:e7:a5:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:f2:e7:a5:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:1d:1c:cd:0d:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 16:33:15.993540 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.993530 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 16:33:15.993760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.993686 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 16:33:15.997232 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.997199 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 16:33:15.997425 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.997235 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 16:33:15.998159 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.998146 2559 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 16:33:15.998212 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.998166 2559 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 16:33:15.998212 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.998176 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 16:33:15.999021 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:15.999007 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 16:33:16.000354 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.000340 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 16:33:16.004791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.004773 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 08 16:33:16.004950 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.004938 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 16:33:16.005013 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.004982 2559 kubelet.go:486] "Attempting to sync node with API server" Jun 08 16:33:16.005013 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.005006 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 16:33:16.005102 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.005028 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 16:33:16.005102 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.005043 2559 kubelet.go:398] "Adding apiserver pod source" Jun 08 16:33:16.005102 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.005068 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 16:33:16.010010 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.009991 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 16:33:16.010124 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.010021 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 16:33:16.012691 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.012674 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 16:33:16.014481 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.014465 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 16:33:16.014544 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.014496 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 16:33:16.015828 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015816 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015832 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015838 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015851 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015860 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015867 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015875 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015880 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015886 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015892 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 16:33:16.015904 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015904 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 16:33:16.016155 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.015910 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 16:33:16.016358 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.016336 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 16:33:16.016404 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.016336 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 16:33:16.016860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.016850 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 16:33:16.016860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.016860 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 16:33:16.020055 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.020041 2559 server.go:1268] "Started kubelet" Jun 08 16:33:16.020172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.020125 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 16:33:16.020229 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.020148 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 16:33:16.020334 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.020196 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 16:33:16.020977 ip-10-0-141-92 systemd[1]: Started Kubernetes Kubelet. Jun 08 16:33:16.022453 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.022438 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 16:33:16.023773 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.023755 2559 server.go:310] "Adding debug handlers to kubelet server" Jun 08 16:33:16.031237 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.031204 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 16:33:16.031902 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.031883 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 16:33:16.033109 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.031906 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-92.ec2.internal.18b728376456518b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-92.ec2.internal,UID:ip-10-0-141-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-92.ec2.internal,},FirstTimestamp:2026-06-08 16:33:16.020011403 +0000 UTC m=+0.406199787,LastTimestamp:2026-06-08 16:33:16.020011403 +0000 UTC m=+0.406199787,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-92.ec2.internal,}" Jun 08 16:33:16.033179 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.033158 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 16:33:16.033226 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.033179 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 16:33:16.033324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.033297 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.033941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.033912 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 16:33:16.033941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.033940 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 16:33:16.034066 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.033914 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 16:33:16.034066 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034048 2559 reconstruct.go:97] "Volume reconstruction finished" Jun 08 16:33:16.034066 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034054 2559 reconciler.go:29] "Reconciler: start to sync state" Jun 08 16:33:16.034158 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034138 2559 factory.go:55] Registering systemd factory Jun 08 16:33:16.034203 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034185 2559 factory.go:223] Registration of the systemd container factory successfully Jun 08 16:33:16.034511 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034495 2559 factory.go:153] Registering CRI-O factory Jun 08 16:33:16.034568 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034514 2559 factory.go:223] Registration of the crio container factory successfully Jun 08 16:33:16.034568 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034558 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 16:33:16.034676 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034580 2559 factory.go:103] Registering Raw factory Jun 08 16:33:16.034676 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034594 2559 manager.go:1196] Started watching for new ooms in manager Jun 08 16:33:16.035012 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.034997 2559 manager.go:349] Starting recovery of all containers Jun 08 16:33:16.035085 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.035047 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 08 16:33:16.035125 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.035073 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 08 16:33:16.045302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.045119 2559 manager.go:354] Recovery completed Jun 08 16:33:16.049290 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.049276 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.050147 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.050128 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jwb5q" Jun 08 16:33:16.051282 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051269 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.051330 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051302 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.051330 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051315 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.051899 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051882 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 16:33:16.051973 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051898 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.051973 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051899 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 16:33:16.051973 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.051959 2559 state_mem.go:36] "Initialized new in-memory state store" Jun 08 16:33:16.052534 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.052517 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.052592 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.052550 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.052592 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.052565 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.054088 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.054076 2559 policy_none.go:49] "None policy: Start" Jun 08 16:33:16.054133 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.054093 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 16:33:16.054133 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.054107 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 16:33:16.055786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.055775 2559 policy_none.go:47] "Start" Jun 08 16:33:16.056229 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.056215 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jwb5q" Jun 08 16:33:16.060177 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.060057 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-92.ec2.internal.18b7283766337ea0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-92.ec2.internal,UID:ip-10-0-141-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-92.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-92.ec2.internal,},FirstTimestamp:2026-06-08 16:33:16.051283616 +0000 UTC m=+0.437472003,LastTimestamp:2026-06-08 16:33:16.051283616 +0000 UTC m=+0.437472003,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-92.ec2.internal,}" Jun 08 16:33:16.095347 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095325 2559 manager.go:337] "Starting Device Plugin manager" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.095364 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095377 2559 server.go:87] "Starting device plugin registration server" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095695 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095708 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095792 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095867 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.095873 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.096489 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.096543 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.110177 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.111478 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.111528 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.111554 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.111597 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 16:33:16.118379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.114017 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 16:33:16.196372 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.196281 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.198581 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.198552 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.198581 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.198586 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.198762 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.198598 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.198762 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.198621 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.208144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.208107 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.208144 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.208145 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-92.ec2.internal\": node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.212086 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.212057 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal"] Jun 08 16:33:16.212208 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.212131 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.213022 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.213005 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.213110 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.213042 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.213110 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.213057 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.215650 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.215619 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.215737 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.215724 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.215791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.215753 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.216372 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216358 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.216442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216359 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.216442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216387 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.216442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216396 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.216442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216411 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.216442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.216423 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.218689 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.218672 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.218746 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.218705 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 16:33:16.219426 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.219404 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientMemory" Jun 08 16:33:16.219518 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.219435 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 16:33:16.219518 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.219448 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeHasSufficientPID" Jun 08 16:33:16.235260 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.235225 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f5161e243f7f1549af0b115bd8013b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-92.ec2.internal\" (UID: \"c4f5161e243f7f1549af0b115bd8013b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.235260 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.235254 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.235446 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.235271 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.236857 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.236838 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.241475 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.241454 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-92.ec2.internal\" not found" node="ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.246252 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.246235 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-92.ec2.internal\" not found" node="ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336242 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336206 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f5161e243f7f1549af0b115bd8013b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-92.ec2.internal\" (UID: \"c4f5161e243f7f1549af0b115bd8013b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336242 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336304 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f5161e243f7f1549af0b115bd8013b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-92.ec2.internal\" (UID: \"c4f5161e243f7f1549af0b115bd8013b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.336423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.336350 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/32875e280a5f2bf9361e6c044736beca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal\" (UID: \"32875e280a5f2bf9361e6c044736beca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.337268 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.337253 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.438325 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.438279 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.539136 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.539067 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.548237 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.548211 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.553700 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.553680 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:16.639597 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.639553 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.740094 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.740062 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.840559 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.840474 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.940868 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:16.940832 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:16.960030 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.960000 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 16:33:16.960189 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:16.960172 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 16:33:17.041107 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:17.041084 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-92.ec2.internal\" not found" Jun 08 16:33:17.052210 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:17.051239 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f5161e243f7f1549af0b115bd8013b.slice/crio-fb8ab40276e747201febe21834dc4d6220389debd51b89233d45705f42a1fc5a WatchSource:0}: Error finding container fb8ab40276e747201febe21834dc4d6220389debd51b89233d45705f42a1fc5a: Status 404 returned error can't find the container with id fb8ab40276e747201febe21834dc4d6220389debd51b89233d45705f42a1fc5a Jun 08 16:33:17.052363 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:17.052307 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32875e280a5f2bf9361e6c044736beca.slice/crio-853c4e459c076029814c682da4a949e14411abb795ea69860348a1d436431701 WatchSource:0}: Error finding container 853c4e459c076029814c682da4a949e14411abb795ea69860348a1d436431701: Status 404 returned error can't find the container with id 853c4e459c076029814c682da4a949e14411abb795ea69860348a1d436431701 Jun 08 16:33:17.056999 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.056975 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:33:17.059618 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.059593 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 16:28:16 +0000 UTC" deadline="2027-12-22 13:43:05.741653586 +0000 UTC" Jun 08 16:33:17.059698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.059618 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13485h9m48.682037878s" Jun 08 16:33:17.115114 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.115003 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" event={"ID":"c4f5161e243f7f1549af0b115bd8013b","Type":"ContainerStarted","Data":"fb8ab40276e747201febe21834dc4d6220389debd51b89233d45705f42a1fc5a"} Jun 08 16:33:17.116085 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.116057 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" event={"ID":"32875e280a5f2bf9361e6c044736beca","Type":"ContainerStarted","Data":"853c4e459c076029814c682da4a949e14411abb795ea69860348a1d436431701"} Jun 08 16:33:17.116363 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.116349 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 16:33:17.133766 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.133743 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" Jun 08 16:33:17.147501 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.147479 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 16:33:17.149521 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.149506 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" Jun 08 16:33:17.159364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.159342 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 16:33:17.233799 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.233767 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 16:33:17.574817 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:17.574722 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 16:33:18.006037 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.005948 2559 apiserver.go:52] "Watching apiserver" Jun 08 16:33:18.016869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.016833 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 16:33:18.017255 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.017222 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-bd2x9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal","openshift-multus/multus-pmwqs","kube-system/global-pull-secret-syncer-lq2k9","openshift-multus/multus-additional-cni-plugins-6qkwf","openshift-multus/network-metrics-daemon-5z57p","openshift-network-diagnostics/network-check-target-wtxrd","openshift-network-operator/iptables-alerter-98flx","openshift-ovn-kubernetes/ovnkube-node-tlf2q","kube-system/konnectivity-agent-zgjvf","kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8","openshift-cluster-node-tuning-operator/tuned-wbmlx","openshift-dns/node-resolver-kxw8p"] Jun 08 16:33:18.020125 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.020094 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:18.020258 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.020176 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:18.022233 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.022207 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.025791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.024375 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.025791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.024893 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.025791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.024904 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.025791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.025082 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c8njz\"" Jun 08 16:33:18.027014 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.026993 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.027132 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.027054 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:18.027803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.027565 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 16:33:18.027803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.027650 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 16:33:18.028319 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.028158 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.028319 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.028203 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vb6nh\"" Jun 08 16:33:18.028556 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.028454 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.029759 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.029541 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.032114 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.032093 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 16:33:18.032223 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.032193 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 16:33:18.032287 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.032198 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bnz7p\"" Jun 08 16:33:18.032651 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.032620 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.032779 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.032730 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:18.034975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.034427 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 16:33:18.035137 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.035111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.037751 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.037729 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.037858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.037800 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 16:33:18.037914 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.037896 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.037970 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.037800 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ggr8s\"" Jun 08 16:33:18.039345 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.039045 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.039345 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.039205 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 16:33:18.040033 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.040012 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 16:33:18.041172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.040482 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 16:33:18.041172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.040554 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.041172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.040781 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.041172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.040982 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-th77h\"" Jun 08 16:33:18.041172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.041041 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 16:33:18.041478 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.041276 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.042685 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.042623 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.043813 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.043792 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.044094 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.044042 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q6dlg\"" Jun 08 16:33:18.044182 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.044133 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.045201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045167 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.045393 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045375 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 16:33:18.045495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045413 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 16:33:18.045724 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045700 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lkgbb\"" Jun 08 16:33:18.045950 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045928 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-conf\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.046052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmwb6\" (UniqueName: \"kubernetes.io/projected/6e842aae-0a40-4404-86ce-a3083951991b-kube-api-access-vmwb6\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.046052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.045989 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-slash\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.046052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-config\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.046189 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-os-release\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.046189 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-netns\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.046189 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-ovn\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.046314 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046201 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-os-release\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.046314 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046244 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.046825 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-env-overrides\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.046921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046849 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.046921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-host\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.046921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046912 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.047083 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.046936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-cni-binary-copy\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.047083 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047000 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.047227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047037 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-modprobe-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.047227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-lib-modules\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.047336 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047250 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-tmp\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.047336 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-socket-dir-parent\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.047436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-etc-kubernetes\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.047436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047384 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh6h6\" (UniqueName: \"kubernetes.io/projected/f972084f-acf6-4427-b7ea-0f340e15343b-kube-api-access-lh6h6\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.047436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-var-lib-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.047601 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysconfig\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.047601 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047519 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-systemd\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.047601 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2mvx\" (UniqueName: \"kubernetes.io/projected/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-kube-api-access-d2mvx\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.047743 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047611 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-kubelet-config\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.047884 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.047832 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.048095 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.048040 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.048497 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.048382 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vjspv\"" Jun 08 16:33:18.049062 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049043 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 16:33:18.049433 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049415 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.049794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049757 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-etc-tuned\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049824 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049830 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049863 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.049965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-netns\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050002 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-systemd-units\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-systemd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050153 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050061 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2chw\" (UniqueName: \"kubernetes.io/projected/5ed14746-301c-4904-875c-8ba3415aaf7c-kube-api-access-p2chw\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050332 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-system-cni-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.050476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-multus-certs\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.050476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-kubelet\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-script-lib\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-run\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.050616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050570 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-var-lib-kubelet\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.050777 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050650 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-bin\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050777 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ed14746-301c-4904-875c-8ba3415aaf7c-ovn-node-metrics-cert\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.050777 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050741 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:18.050908 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050801 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cnibin\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.050908 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-hostroot\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.050908 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050862 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-conf-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050906 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbdp\" (UniqueName: \"kubernetes.io/projected/cca56a36-b98a-4080-a137-5aa7848aaca7-kube-api-access-frbdp\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050934 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-dbus\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050956 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-system-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050974 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-cnibin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.050998 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-bin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-multus\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051044 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-daemon-config\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051063 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-kubernetes\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-log-socket\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051112 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-netd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051116 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051141 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051165 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-etc-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-sys\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051270 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.051352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051297 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-k8s-cni-cncf-io\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051857 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-node-log\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051857 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.051857 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051656 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-kubelet\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.051857 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051733 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rwjpj\"" Jun 08 16:33:18.052063 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.051874 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 16:33:18.052115 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.052064 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 16:33:18.067944 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.067919 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcl57" Jun 08 16:33:18.074886 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.074863 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcl57" Jun 08 16:33:18.134646 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.134602 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 16:33:18.152808 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152770 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-conf\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmwb6\" (UniqueName: \"kubernetes.io/projected/6e842aae-0a40-4404-86ce-a3083951991b-kube-api-access-vmwb6\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-slash\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-config\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152910 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-os-release\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152942 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-device-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152968 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-netns\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152974 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-conf\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.152994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152994 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-ovn\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153051 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-os-release\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153039 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-os-release\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153092 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-ovn\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153093 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a14650-8507-4ce8-8301-987731013a88-host\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153172 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a14650-8507-4ce8-8301-987731013a88-serviceca\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153106 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-os-release\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.152980 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-slash\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153059 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-netns\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153196 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7ln\" (UniqueName: \"kubernetes.io/projected/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-kube-api-access-tg7ln\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153228 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-env-overrides\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153366 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.153298 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.153432 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:18.653411208 +0000 UTC m=+3.039599593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysctl-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153484 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-host\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153505 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-tmp-dir\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153514 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-config\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153521 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-socket-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153564 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-host\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153576 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-env-overrides\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153628 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-cni-binary-copy\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153700 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-modprobe-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153710 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153721 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-lib-modules\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153736 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-tmp\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.153754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153743 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153753 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-sys-fs\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-socket-dir-parent\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153802 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-etc-kubernetes\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh6h6\" (UniqueName: \"kubernetes.io/projected/f972084f-acf6-4427-b7ea-0f340e15343b-kube-api-access-lh6h6\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153830 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-lib-modules\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-iptables-alerter-script\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153867 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ftz9\" (UniqueName: \"kubernetes.io/projected/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-kube-api-access-8ftz9\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153881 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-modprobe-d\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153887 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/724a94f7-5d35-4fe6-be48-2da68cfa3f54-konnectivity-ca\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153907 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-etc-kubernetes\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153954 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-var-lib-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153973 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysconfig\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.153988 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-systemd\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154007 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2mvx\" (UniqueName: \"kubernetes.io/projected/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-kube-api-access-d2mvx\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-kubelet-config\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154046 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-hosts-file\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.154201 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154061 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-etc-tuned\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154092 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154107 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154111 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-netns\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjt4m\" (UniqueName: \"kubernetes.io/projected/40a14650-8507-4ce8-8301-987731013a88-kube-api-access-xjt4m\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154164 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-systemd-units\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-systemd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2chw\" (UniqueName: \"kubernetes.io/projected/5ed14746-301c-4904-875c-8ba3415aaf7c-kube-api-access-p2chw\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-system-cni-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154246 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-multus-certs\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154259 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154254 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-var-lib-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154282 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-kubelet\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-script-lib\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-socket-dir-parent\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154316 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-run\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.154838 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154366 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-run\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154405 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-system-cni-dir\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154428 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-netns\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154444 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-systemd\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154468 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-sysconfig\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154493 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-kubelet-config\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-systemd-units\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154046 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-cni-binary-copy\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154552 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-multus-certs\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154589 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-var-lib-kubelet\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154675 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-host-slash\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154677 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-kubelet\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154702 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-bin\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154713 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-var-lib-kubelet\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154729 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ed14746-301c-4904-875c-8ba3415aaf7c-ovn-node-metrics-cert\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154780 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.155794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cnibin\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154814 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cnibin\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154850 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-hostroot\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154875 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-conf-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154906 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-bin\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154910 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-systemd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154949 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbdp\" (UniqueName: \"kubernetes.io/projected/cca56a36-b98a-4080-a137-5aa7848aaca7-kube-api-access-frbdp\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154963 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-hostroot\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.154976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-dbus\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155000 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-conf-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-system-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155068 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-cnibin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155104 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-binary-copy\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-bin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155173 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-cnibin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155178 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/93ef7e77-f153-47bb-bd69-2a073996344d-dbus\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155204 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-multus\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-bin\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.156598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-daemon-config\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155277 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-system-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155300 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-cni-multus\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155547 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-registration-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155586 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-kubernetes\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155604 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5ed14746-301c-4904-875c-8ba3415aaf7c-ovnkube-script-lib\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-log-socket\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-netd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155687 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/724a94f7-5d35-4fe6-be48-2da68cfa3f54-agent-certs\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155692 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-log-socket\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155711 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c924l\" (UniqueName: \"kubernetes.io/projected/f97a4826-f39e-4b4c-929d-a243d140116f-kube-api-access-c924l\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155688 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-etc-kubernetes\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155719 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-cni-netd\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155795 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155796 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-run-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.157451 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155842 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-cni-dir\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155844 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-etc-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.155916 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155882 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-etc-openvswitch\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.155965 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:18.655948744 +0000 UTC m=+3.042137131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-sys\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.155915 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e842aae-0a40-4404-86ce-a3083951991b-sys\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156060 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca56a36-b98a-4080-a137-5aa7848aaca7-multus-daemon-config\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156100 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-k8s-cni-cncf-io\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156065 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-run-k8s-cni-cncf-io\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156138 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156165 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-node-log\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-kubelet\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156293 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156297 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5ed14746-301c-4904-875c-8ba3415aaf7c-node-log\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.158513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca56a36-b98a-4080-a137-5aa7848aaca7-host-var-lib-kubelet\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.159361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.156469 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.159361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.158216 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-tmp\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.159361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.158240 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e842aae-0a40-4404-86ce-a3083951991b-etc-tuned\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.159361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.158350 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5ed14746-301c-4904-875c-8ba3415aaf7c-ovn-node-metrics-cert\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.166763 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.166736 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:18.166763 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.166768 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:18.166956 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.166782 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:18.166956 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.166846 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:18.666824889 +0000 UTC m=+3.053013281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:18.168319 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.168226 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmwb6\" (UniqueName: \"kubernetes.io/projected/6e842aae-0a40-4404-86ce-a3083951991b-kube-api-access-vmwb6\") pod \"tuned-wbmlx\" (UID: \"6e842aae-0a40-4404-86ce-a3083951991b\") " pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.168444 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.168397 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh6h6\" (UniqueName: \"kubernetes.io/projected/f972084f-acf6-4427-b7ea-0f340e15343b-kube-api-access-lh6h6\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.169928 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.169904 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2chw\" (UniqueName: \"kubernetes.io/projected/5ed14746-301c-4904-875c-8ba3415aaf7c-kube-api-access-p2chw\") pod \"ovnkube-node-tlf2q\" (UID: \"5ed14746-301c-4904-875c-8ba3415aaf7c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.170346 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.170283 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2mvx\" (UniqueName: \"kubernetes.io/projected/4f962b4e-936c-4cac-a9f3-687fd8bd1c66-kube-api-access-d2mvx\") pod \"multus-additional-cni-plugins-6qkwf\" (UID: \"4f962b4e-936c-4cac-a9f3-687fd8bd1c66\") " pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.170346 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.170324 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbdp\" (UniqueName: \"kubernetes.io/projected/cca56a36-b98a-4080-a137-5aa7848aaca7-kube-api-access-frbdp\") pod \"multus-pmwqs\" (UID: \"cca56a36-b98a-4080-a137-5aa7848aaca7\") " pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.257249 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257160 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-registration-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257249 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/724a94f7-5d35-4fe6-be48-2da68cfa3f54-agent-certs\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.257249 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257239 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c924l\" (UniqueName: \"kubernetes.io/projected/f97a4826-f39e-4b4c-929d-a243d140116f-kube-api-access-c924l\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257309 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257321 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-registration-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257344 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-device-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257382 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a14650-8507-4ce8-8301-987731013a88-host\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257395 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257404 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a14650-8507-4ce8-8301-987731013a88-serviceca\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257427 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7ln\" (UniqueName: \"kubernetes.io/projected/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-kube-api-access-tg7ln\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257440 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-device-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257451 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-tmp-dir\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-socket-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.257519 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257509 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-sys-fs\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257536 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-iptables-alerter-script\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257560 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ftz9\" (UniqueName: \"kubernetes.io/projected/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-kube-api-access-8ftz9\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257583 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/724a94f7-5d35-4fe6-be48-2da68cfa3f54-konnectivity-ca\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257611 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-hosts-file\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257662 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjt4m\" (UniqueName: \"kubernetes.io/projected/40a14650-8507-4ce8-8301-987731013a88-kube-api-access-xjt4m\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257692 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-host-slash\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-host-slash\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-tmp-dir\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257857 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a14650-8507-4ce8-8301-987731013a88-host\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257943 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-socket-dir\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.257950 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a14650-8507-4ce8-8301-987731013a88-serviceca\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.258050 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-etc-selinux\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.258164 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-hosts-file\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.258218 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f97a4826-f39e-4b4c-929d-a243d140116f-sys-fs\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.258840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.258405 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-iptables-alerter-script\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.259591 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.259074 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/724a94f7-5d35-4fe6-be48-2da68cfa3f54-konnectivity-ca\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.262386 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.262362 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/724a94f7-5d35-4fe6-be48-2da68cfa3f54-agent-certs\") pod \"konnectivity-agent-zgjvf\" (UID: \"724a94f7-5d35-4fe6-be48-2da68cfa3f54\") " pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.266438 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.266411 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7ln\" (UniqueName: \"kubernetes.io/projected/5aa1fc49-eeb7-46c3-9a73-4511f4cc7383-kube-api-access-tg7ln\") pod \"iptables-alerter-98flx\" (UID: \"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383\") " pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.266561 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.266415 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjt4m\" (UniqueName: \"kubernetes.io/projected/40a14650-8507-4ce8-8301-987731013a88-kube-api-access-xjt4m\") pod \"node-ca-bd2x9\" (UID: \"40a14650-8507-4ce8-8301-987731013a88\") " pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.266625 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.266604 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ftz9\" (UniqueName: \"kubernetes.io/projected/1dedd458-09c2-40f1-9dfa-2bd02e1f8965-kube-api-access-8ftz9\") pod \"node-resolver-kxw8p\" (UID: \"1dedd458-09c2-40f1-9dfa-2bd02e1f8965\") " pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.266689 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.266609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c924l\" (UniqueName: \"kubernetes.io/projected/f97a4826-f39e-4b4c-929d-a243d140116f-kube-api-access-c924l\") pod \"aws-ebs-csi-driver-node-gd9p8\" (UID: \"f97a4826-f39e-4b4c-929d-a243d140116f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.300377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.300346 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 16:33:18.335933 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.335899 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" Jun 08 16:33:18.343712 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.343686 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pmwqs" Jun 08 16:33:18.356470 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.356435 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" Jun 08 16:33:18.360497 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.360472 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:18.369225 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.369199 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bd2x9" Jun 08 16:33:18.376966 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.376938 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kxw8p" Jun 08 16:33:18.382778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.382754 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:18.390482 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.390460 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-98flx" Jun 08 16:33:18.395258 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.395235 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" Jun 08 16:33:18.660302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.660212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:18.660302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.660267 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:18.660464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.660339 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:18.660464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.660371 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:18.660464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.660409 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:19.660392309 +0000 UTC m=+4.046580701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:18.660464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.660424 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:19.66041797 +0000 UTC m=+4.046606342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:18.761428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:18.761391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:18.761587 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.761523 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:18.761587 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.761540 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:18.761587 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.761553 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:18.761766 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:18.761603 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:19.761590677 +0000 UTC m=+4.147779049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:19.018541 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.018506 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dedd458_09c2_40f1_9dfa_2bd02e1f8965.slice/crio-4453840f4cd77e299b8299be53e2fb9f6d6402e0405ba9466774d82eceb12a37 WatchSource:0}: Error finding container 4453840f4cd77e299b8299be53e2fb9f6d6402e0405ba9466774d82eceb12a37: Status 404 returned error can't find the container with id 4453840f4cd77e299b8299be53e2fb9f6d6402e0405ba9466774d82eceb12a37 Jun 08 16:33:19.020152 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.020109 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f962b4e_936c_4cac_a9f3_687fd8bd1c66.slice/crio-19447ff2f2f17041730025fff4853b17543ee34de4a0f050833b6e7d457092bd WatchSource:0}: Error finding container 19447ff2f2f17041730025fff4853b17543ee34de4a0f050833b6e7d457092bd: Status 404 returned error can't find the container with id 19447ff2f2f17041730025fff4853b17543ee34de4a0f050833b6e7d457092bd Jun 08 16:33:19.021046 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.021027 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca56a36_b98a_4080_a137_5aa7848aaca7.slice/crio-74b10157d0d63fc380b42227e04af26ac0c9c5005f6887e146bbc25fd892e18e WatchSource:0}: Error finding container 74b10157d0d63fc380b42227e04af26ac0c9c5005f6887e146bbc25fd892e18e: Status 404 returned error can't find the container with id 74b10157d0d63fc380b42227e04af26ac0c9c5005f6887e146bbc25fd892e18e Jun 08 16:33:19.044018 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.043983 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed14746_301c_4904_875c_8ba3415aaf7c.slice/crio-f1f4d3024431b9cc1e8011b3ee2630864e863c659351b6ec8c1e800944801fe6 WatchSource:0}: Error finding container f1f4d3024431b9cc1e8011b3ee2630864e863c659351b6ec8c1e800944801fe6: Status 404 returned error can't find the container with id f1f4d3024431b9cc1e8011b3ee2630864e863c659351b6ec8c1e800944801fe6 Jun 08 16:33:19.047168 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.047136 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a14650_8507_4ce8_8301_987731013a88.slice/crio-a32d778db47d478fce5450d68694afd4587c00ca921fdb51e5c26fe559b5b328 WatchSource:0}: Error finding container a32d778db47d478fce5450d68694afd4587c00ca921fdb51e5c26fe559b5b328: Status 404 returned error can't find the container with id a32d778db47d478fce5450d68694afd4587c00ca921fdb51e5c26fe559b5b328 Jun 08 16:33:19.048708 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.048685 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa1fc49_eeb7_46c3_9a73_4511f4cc7383.slice/crio-ecaa0bfb0c728c244daa26bd3cf781f5f8ed3479a380a4b69311ea98523f14ea WatchSource:0}: Error finding container ecaa0bfb0c728c244daa26bd3cf781f5f8ed3479a380a4b69311ea98523f14ea: Status 404 returned error can't find the container with id ecaa0bfb0c728c244daa26bd3cf781f5f8ed3479a380a4b69311ea98523f14ea Jun 08 16:33:19.049853 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.049819 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf97a4826_f39e_4b4c_929d_a243d140116f.slice/crio-6c541acea5eec317e732d0b609a377582781f141225dc98ebcb4a598fe120a95 WatchSource:0}: Error finding container 6c541acea5eec317e732d0b609a377582781f141225dc98ebcb4a598fe120a95: Status 404 returned error can't find the container with id 6c541acea5eec317e732d0b609a377582781f141225dc98ebcb4a598fe120a95 Jun 08 16:33:19.051241 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.051219 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod724a94f7_5d35_4fe6_be48_2da68cfa3f54.slice/crio-cf479a80990d418a64d70ec11ddd03dd4e5b9758da7f99bf1d26478bfe46bfa3 WatchSource:0}: Error finding container cf479a80990d418a64d70ec11ddd03dd4e5b9758da7f99bf1d26478bfe46bfa3: Status 404 returned error can't find the container with id cf479a80990d418a64d70ec11ddd03dd4e5b9758da7f99bf1d26478bfe46bfa3 Jun 08 16:33:19.052658 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:19.052203 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e842aae_0a40_4404_86ce_a3083951991b.slice/crio-47f12d9687a307c0a615a23d364fbfdccb18e96bae2f93ab57c8ac83f73d5376 WatchSource:0}: Error finding container 47f12d9687a307c0a615a23d364fbfdccb18e96bae2f93ab57c8ac83f73d5376: Status 404 returned error can't find the container with id 47f12d9687a307c0a615a23d364fbfdccb18e96bae2f93ab57c8ac83f73d5376 Jun 08 16:33:19.078174 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.078013 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 16:28:18 +0000 UTC" deadline="2028-01-16 00:38:52.132394026 +0000 UTC" Jun 08 16:33:19.078174 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.078172 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14072h5m33.054224893s" Jun 08 16:33:19.120144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.120111 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kxw8p" event={"ID":"1dedd458-09c2-40f1-9dfa-2bd02e1f8965","Type":"ContainerStarted","Data":"4453840f4cd77e299b8299be53e2fb9f6d6402e0405ba9466774d82eceb12a37"} Jun 08 16:33:19.121224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.121200 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" event={"ID":"6e842aae-0a40-4404-86ce-a3083951991b","Type":"ContainerStarted","Data":"47f12d9687a307c0a615a23d364fbfdccb18e96bae2f93ab57c8ac83f73d5376"} Jun 08 16:33:19.122221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.122197 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgjvf" event={"ID":"724a94f7-5d35-4fe6-be48-2da68cfa3f54","Type":"ContainerStarted","Data":"cf479a80990d418a64d70ec11ddd03dd4e5b9758da7f99bf1d26478bfe46bfa3"} Jun 08 16:33:19.123088 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.123069 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" event={"ID":"f97a4826-f39e-4b4c-929d-a243d140116f","Type":"ContainerStarted","Data":"6c541acea5eec317e732d0b609a377582781f141225dc98ebcb4a598fe120a95"} Jun 08 16:33:19.124145 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.124121 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98flx" event={"ID":"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383","Type":"ContainerStarted","Data":"ecaa0bfb0c728c244daa26bd3cf781f5f8ed3479a380a4b69311ea98523f14ea"} Jun 08 16:33:19.125170 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.125142 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bd2x9" event={"ID":"40a14650-8507-4ce8-8301-987731013a88","Type":"ContainerStarted","Data":"a32d778db47d478fce5450d68694afd4587c00ca921fdb51e5c26fe559b5b328"} Jun 08 16:33:19.126337 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.126319 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"f1f4d3024431b9cc1e8011b3ee2630864e863c659351b6ec8c1e800944801fe6"} Jun 08 16:33:19.127465 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.127444 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmwqs" event={"ID":"cca56a36-b98a-4080-a137-5aa7848aaca7","Type":"ContainerStarted","Data":"74b10157d0d63fc380b42227e04af26ac0c9c5005f6887e146bbc25fd892e18e"} Jun 08 16:33:19.128488 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.128463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerStarted","Data":"19447ff2f2f17041730025fff4853b17543ee34de4a0f050833b6e7d457092bd"} Jun 08 16:33:19.668715 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.668609 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:19.668715 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.668693 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:19.668940 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.668820 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:19.668940 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.668882 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:21.668863817 +0000 UTC m=+6.055052195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:19.669061 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.668954 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:19.669061 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.668990 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:21.668977901 +0000 UTC m=+6.055166277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:19.769901 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:19.769852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:19.770155 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.770095 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:19.770155 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.770119 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:19.770155 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.770131 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:19.770318 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:19.770193 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:21.770174697 +0000 UTC m=+6.156363087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:20.078548 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.078432 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 16:28:18 +0000 UTC" deadline="2027-12-11 15:53:29.190285695 +0000 UTC" Jun 08 16:33:20.078548 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.078500 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13223h20m9.111790172s" Jun 08 16:33:20.112623 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.111976 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:20.112623 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:20.112099 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:20.112623 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.112478 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:20.112623 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:20.112577 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:20.112623 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.112620 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:20.112998 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:20.112723 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:20.145454 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.145410 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" event={"ID":"c4f5161e243f7f1549af0b115bd8013b","Type":"ContainerStarted","Data":"1920f4b5f9e9d51192b38e384daabbb1a5b2339a93b3895a5009a40341994ca6"} Jun 08 16:33:20.153286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.153252 2559 generic.go:356] "Generic (PLEG): container finished" podID="32875e280a5f2bf9361e6c044736beca" containerID="63d86e0d80c2a4001d0514c1ccaf1aea014b83dbc9adb1014deb39bf19c7daf6" exitCode=0 Jun 08 16:33:20.153456 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.153304 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" event={"ID":"32875e280a5f2bf9361e6c044736beca","Type":"ContainerDied","Data":"63d86e0d80c2a4001d0514c1ccaf1aea014b83dbc9adb1014deb39bf19c7daf6"} Jun 08 16:33:20.183609 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:20.183542 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-92.ec2.internal" podStartSLOduration=3.183516672 podStartE2EDuration="3.183516672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:20.16567337 +0000 UTC m=+4.551861763" watchObservedRunningTime="2026-06-08 16:33:20.183516672 +0000 UTC m=+4.569705068" Jun 08 16:33:21.216744 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:21.216703 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" event={"ID":"32875e280a5f2bf9361e6c044736beca","Type":"ContainerStarted","Data":"14555be89bb51a566e879b3393d6fe40fb2a31cc514dab05784305fae5e93ffa"} Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:21.687384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:21.687485 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.687611 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.687694 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:25.687673571 +0000 UTC m=+10.073861960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.688090 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:21.688324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.688174 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:25.688124845 +0000 UTC m=+10.074313231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:21.789355 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:21.788681 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:21.789355 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.788911 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:21.789355 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.788935 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:21.789355 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.788948 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:21.789355 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:21.789012 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:25.78899275 +0000 UTC m=+10.175181138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:22.112833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:22.112750 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:22.112833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:22.112777 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:22.113044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:22.112909 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:22.113044 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:22.112900 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:22.113044 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:22.112996 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:22.113196 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:22.113067 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:24.114662 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:24.113651 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:24.114662 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:24.113792 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:24.114662 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:24.114233 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:24.114662 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:24.114338 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:24.115291 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:24.114799 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:24.115291 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:24.114902 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:25.729429 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:25.729388 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:25.729429 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:25.729432 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:25.729937 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.729524 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:25.729937 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.729625 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:33.729609081 +0000 UTC m=+18.115797453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:25.729937 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.729779 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:25.729937 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.729847 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:33.729830141 +0000 UTC m=+18.116018516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:25.830034 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:25.829993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:25.830220 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.830190 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:25.830220 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.830213 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:25.830324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.830226 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:25.830324 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:25.830288 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:33.83026876 +0000 UTC m=+18.216457149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:26.113184 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:26.113095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:26.113353 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:26.113205 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:26.113651 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:26.113476 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:26.113651 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:26.113588 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:26.113802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:26.113651 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:26.113802 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:26.113763 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:28.112790 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:28.112753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:28.113245 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:28.112888 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:28.113376 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:28.113324 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:28.113678 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:28.113487 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:28.113678 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:28.113557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:28.113678 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:28.113621 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:30.113781 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:30.113746 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:30.114261 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:30.113743 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:30.114261 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:30.113876 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:30.114261 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:30.113748 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:30.114261 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:30.113935 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:30.114261 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:30.114054 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:32.112406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:32.112372 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:32.112831 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:32.112385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:32.112831 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:32.112487 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:32.112831 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:32.112519 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:32.112995 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:32.112626 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:32.116322 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:32.116283 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:33.789663 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:33.789565 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:33.789663 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:33.789622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:33.790182 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.789735 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:33.790182 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.789739 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:33.790182 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.789795 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:33:49.78978085 +0000 UTC m=+34.175969222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:33.790182 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.789814 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:33:49.789804226 +0000 UTC m=+34.175992598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:33.890505 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:33.890460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:33.890728 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.890618 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:33.890728 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.890662 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:33.890728 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.890676 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:33.890884 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:33.890744 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:49.890725096 +0000 UTC m=+34.276913489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:34.112100 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:34.112006 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:34.112100 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:34.112081 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:34.112100 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:34.112082 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:34.112372 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:34.112184 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:34.112372 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:34.112262 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:34.112372 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:34.112354 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:36.112886 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:36.112852 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:36.113343 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:36.112938 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:36.113343 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:36.112970 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:36.113343 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:36.112988 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:36.113343 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:36.113042 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:36.113343 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:36.113143 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:38.113286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.112558 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:38.113286 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:38.112894 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:38.113286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.113253 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:38.113905 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:38.113367 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:38.113905 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.113557 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:38.113905 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:38.113629 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:38.252052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.251708 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" event={"ID":"6e842aae-0a40-4404-86ce-a3083951991b","Type":"ContainerStarted","Data":"418cf3be0db09d3f6c1da1bd433f486d18379f96bb10017b0226847441c9715d"} Jun 08 16:33:38.254293 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.254145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgjvf" event={"ID":"724a94f7-5d35-4fe6-be48-2da68cfa3f54","Type":"ContainerStarted","Data":"d6d81581ab31ac526de7ef3d634e0dfbde4333ea2db98eb47b03771a2cc8232a"} Jun 08 16:33:38.256734 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.256706 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" event={"ID":"f97a4826-f39e-4b4c-929d-a243d140116f","Type":"ContainerStarted","Data":"dfbd8a36691674d205133a456e395934e97ec25c76b7f73e3e80806dcdbf0bec"} Jun 08 16:33:38.258499 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.258468 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bd2x9" event={"ID":"40a14650-8507-4ce8-8301-987731013a88","Type":"ContainerStarted","Data":"6bba1634a4fc983b8b22a554927b867eb88da6073b61e0aa6341059a5d8ecd27"} Jun 08 16:33:38.260606 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.260581 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"e7fbe3ed0ee30e1b32bc7a5a9cbbf309c84c2f7cd141ce2fec38ea0cb60fc4f9"} Jun 08 16:33:38.263247 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.263216 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pmwqs" event={"ID":"cca56a36-b98a-4080-a137-5aa7848aaca7","Type":"ContainerStarted","Data":"3e6145d9d27c955714e72dc0bd973f7f043fd9dc2b70e194ac19bb56c3693f7d"} Jun 08 16:33:38.270914 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.270281 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerStarted","Data":"6824da242f1802ff5886788789e6a957e9f3b8c34fd82a22da3451a3c93efd35"} Jun 08 16:33:38.271921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.271880 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-92.ec2.internal" podStartSLOduration=21.271868103 podStartE2EDuration="21.271868103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:21.244459171 +0000 UTC m=+5.630647566" watchObservedRunningTime="2026-06-08 16:33:38.271868103 +0000 UTC m=+22.658056498" Jun 08 16:33:38.272591 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.272566 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kxw8p" event={"ID":"1dedd458-09c2-40f1-9dfa-2bd02e1f8965","Type":"ContainerStarted","Data":"5fd4412ba5aee30a403036f5eb8491e71e1e0a71fc560d5c504b6d20da75d741"} Jun 08 16:33:38.296790 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.296747 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wbmlx" podStartSLOduration=3.40734547 podStartE2EDuration="22.296734179s" totalImagesPullingTime="18.889388709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:38.271799576 +0000 UTC m=+22.657987987" watchObservedRunningTime="2026-06-08 16:33:38.296734179 +0000 UTC m=+22.682922572" Jun 08 16:33:38.300192 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.300170 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:38.300823 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.300804 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:38.319701 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.319649 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bd2x9" podStartSLOduration=3.618439757 podStartE2EDuration="22.319615034s" totalImagesPullingTime="18.701175277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:38.317486505 +0000 UTC m=+22.703674926" watchObservedRunningTime="2026-06-08 16:33:38.319615034 +0000 UTC m=+22.705803429" Jun 08 16:33:38.337014 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.336953 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zgjvf" podStartSLOduration=7.963800422 podStartE2EDuration="22.336937535s" totalImagesPullingTime="14.373137113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:38.336321017 +0000 UTC m=+22.722509412" watchObservedRunningTime="2026-06-08 16:33:38.336937535 +0000 UTC m=+22.723125929" Jun 08 16:33:38.371608 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.371551 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pmwqs" podStartSLOduration=3.424723026 podStartE2EDuration="22.371530421s" totalImagesPullingTime="18.946807395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:38.354734297 +0000 UTC m=+22.740922706" watchObservedRunningTime="2026-06-08 16:33:38.371530421 +0000 UTC m=+22.757718816" Jun 08 16:33:38.391165 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:38.390953 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kxw8p" podStartSLOduration=8.006762835 podStartE2EDuration="22.390932144s" totalImagesPullingTime="14.384169309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:38.389910666 +0000 UTC m=+22.776099051" watchObservedRunningTime="2026-06-08 16:33:38.390932144 +0000 UTC m=+22.777120541" Jun 08 16:33:39.217256 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.217231 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 16:33:39.276349 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.276311 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" event={"ID":"f97a4826-f39e-4b4c-929d-a243d140116f","Type":"ContainerStarted","Data":"b3c9e8feebc04ed213aa006a71b129a514127c05af18b03934243c4f8a829fed"} Jun 08 16:33:39.278939 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.278919 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:33:39.279253 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279233 2559 generic.go:356] "Generic (PLEG): container finished" podID="5ed14746-301c-4904-875c-8ba3415aaf7c" containerID="a31929581556c7bf402a40b58df801e992a463150f26b5a26ae0a20c0122d789" exitCode=1 Jun 08 16:33:39.279295 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279255 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"cfef0b8678f4fd6785311e6133294f08ba06fc5083bbad037196fefe1f5a4ee8"} Jun 08 16:33:39.279347 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279323 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"70dd686d0924655426bb6f951d94f73e1414cdec308ec0192b879c293546be2d"} Jun 08 16:33:39.279399 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279356 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"71db5c3c90d40dfd570c4a6957e25fc861e07f7f48dcf536a3518e9dfb517b66"} Jun 08 16:33:39.279450 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279398 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"03a8383c2465167ba65cfb319217bfb7bcd86d41dba6ceea8ac4f934d59e4e1e"} Jun 08 16:33:39.279450 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.279437 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerDied","Data":"a31929581556c7bf402a40b58df801e992a463150f26b5a26ae0a20c0122d789"} Jun 08 16:33:39.280746 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.280726 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="6824da242f1802ff5886788789e6a957e9f3b8c34fd82a22da3451a3c93efd35" exitCode=0 Jun 08 16:33:39.280842 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:39.280755 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"6824da242f1802ff5886788789e6a957e9f3b8c34fd82a22da3451a3c93efd35"} Jun 08 16:33:40.112921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.112888 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:40.113124 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.112892 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:40.113124 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:40.113018 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:40.113124 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.113042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:40.113255 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:40.113149 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:40.113255 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:40.113229 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:40.114869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.114771 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T16:33:39.217250161Z","UUID":"54a3e75f-6f6c-4bac-a5d7-4aca6070a119","Handler":null,"Name":"","Endpoint":""} Jun 08 16:33:40.116491 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.116466 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 16:33:40.116602 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.116511 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 16:33:40.284467 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.284426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" event={"ID":"f97a4826-f39e-4b4c-929d-a243d140116f","Type":"ContainerStarted","Data":"b3f63bb846d752d763187422fd302883b756f28ab9d6dabec02e6ce82a972d40"} Jun 08 16:33:40.285890 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.285860 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-98flx" event={"ID":"5aa1fc49-eeb7-46c3-9a73-4511f4cc7383","Type":"ContainerStarted","Data":"2f7a27eb6d078c50c0099f6842fd20a1acac08640b93ba6ab3c915af9d6a1a88"} Jun 08 16:33:40.286012 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.285895 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 16:33:40.301282 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.301225 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gd9p8" podStartSLOduration=3.844942077 podStartE2EDuration="24.301200073s" totalImagesPullingTime="20.456257996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:40.3012026 +0000 UTC m=+24.687390992" watchObservedRunningTime="2026-06-08 16:33:40.301200073 +0000 UTC m=+24.687388462" Jun 08 16:33:40.319833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:40.319778 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-98flx" podStartSLOduration=5.426746034 podStartE2EDuration="24.319762347s" totalImagesPullingTime="18.893016313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:40.319275363 +0000 UTC m=+24.705463760" watchObservedRunningTime="2026-06-08 16:33:40.319762347 +0000 UTC m=+24.705950740" Jun 08 16:33:41.291695 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:41.291514 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:33:41.292158 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:41.292081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"419c86c52b604f0e072cee6524a5143f6f82e6ff13ee781003332dda6a087ab1"} Jun 08 16:33:42.112368 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:42.112328 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:42.112557 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:42.112327 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:42.112557 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:42.112445 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:42.112557 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:42.112510 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:42.112557 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:42.112342 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:42.112770 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:42.112669 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:43.301189 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.301012 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:33:43.302666 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.301589 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"202815e39bb8b6406d96ace0a3cf0e9e8b54af1f367b4e6939e5089079bac43a"} Jun 08 16:33:43.302666 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.301969 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:43.302666 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.302083 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:43.302666 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.302238 2559 scope.go:117] "RemoveContainer" containerID="a31929581556c7bf402a40b58df801e992a463150f26b5a26ae0a20c0122d789" Jun 08 16:33:43.320330 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.320303 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:43.320500 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:43.320437 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:44.112315 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:44.112280 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:44.112516 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:44.112329 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:44.112516 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:44.112415 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:44.112746 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:44.112706 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:44.112866 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:44.112814 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:44.112911 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:44.112888 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:45.001505 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.001461 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lq2k9"] Jun 08 16:33:45.001982 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.001666 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:45.001982 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:45.001766 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:45.004711 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.004689 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wtxrd"] Jun 08 16:33:45.004827 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.004763 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:45.004874 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:45.004852 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:45.005284 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.005265 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5z57p"] Jun 08 16:33:45.005406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.005365 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:45.005536 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:45.005494 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:45.309845 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.309820 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:33:45.310227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.310200 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" event={"ID":"5ed14746-301c-4904-875c-8ba3415aaf7c","Type":"ContainerStarted","Data":"8d8fbc6b17d40e59e157bf3ac543e3581074a50911e53dac60e2fe1393c921b6"} Jun 08 16:33:45.310328 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.310293 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 16:33:45.340318 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:45.340244 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" podStartSLOduration=10.35702604 podStartE2EDuration="29.340220188s" totalImagesPullingTime="18.983194148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:45.338412746 +0000 UTC m=+29.724601141" watchObservedRunningTime="2026-06-08 16:33:45.340220188 +0000 UTC m=+29.726408626" Jun 08 16:33:46.314215 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:46.314175 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="d2aece6b9bf4a71cb29e3198ce0525f439bc11ec1d2b717c167ef7c2e9a2832e" exitCode=0 Jun 08 16:33:46.314689 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:46.314266 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"d2aece6b9bf4a71cb29e3198ce0525f439bc11ec1d2b717c167ef7c2e9a2832e"} Jun 08 16:33:46.314689 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:46.314339 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 16:33:47.112079 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.112039 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:47.112253 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.112062 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:47.112253 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.112166 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:47.112253 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:47.112168 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:47.112253 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:47.112243 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:47.112405 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:47.112326 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:47.206822 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.206776 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:47.206976 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.206924 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 16:33:47.207477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.207450 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zgjvf" Jun 08 16:33:47.909880 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.909623 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:33:47.910340 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.910107 2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 16:33:47.924116 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.924071 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" podUID="5ed14746-301c-4904-875c-8ba3415aaf7c" containerName="ovnkube-controller" probeResult="failure" output="" Jun 08 16:33:47.933280 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:47.933250 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" podUID="5ed14746-301c-4904-875c-8ba3415aaf7c" containerName="ovnkube-controller" probeResult="failure" output="" Jun 08 16:33:48.320665 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:48.320565 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="0c576bb4bf586599de5dca96746e340e880b7981f3d6c75ea7051606f0be39b1" exitCode=0 Jun 08 16:33:48.320665 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:48.320600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"0c576bb4bf586599de5dca96746e340e880b7981f3d6c75ea7051606f0be39b1"} Jun 08 16:33:49.112496 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.112456 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:49.113116 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.112508 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:49.113116 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.112467 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:49.113116 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.112578 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wtxrd" podUID="c1c16413-d27a-4a43-a0c0-e40c1c8aea7c" Jun 08 16:33:49.113116 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.112622 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lq2k9" podUID="93ef7e77-f153-47bb-bd69-2a073996344d" Jun 08 16:33:49.113116 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.112773 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:33:49.811662 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.811610 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:49.811824 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.811680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:49.811824 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.811752 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:49.811824 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.811772 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:49.811824 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.811814 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret podName:93ef7e77-f153-47bb-bd69-2a073996344d nodeName:}" failed. No retries permitted until 2026-06-08 16:34:21.811799645 +0000 UTC m=+66.197988017 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret") pod "global-pull-secret-syncer-lq2k9" (UID: "93ef7e77-f153-47bb-bd69-2a073996344d") : object "kube-system"/"original-pull-secret" not registered Jun 08 16:33:49.811956 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.811829 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:34:21.811822073 +0000 UTC m=+66.198010445 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 16:33:49.912774 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:49.912676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:49.912951 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.912863 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 16:33:49.912951 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.912887 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 16:33:49.912951 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.912900 2559 projected.go:196] Error preparing data for projected volume kube-api-access-l9znh for pod openshift-network-diagnostics/network-check-target-wtxrd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:49.913104 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:49.912966 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh podName:c1c16413-d27a-4a43-a0c0-e40c1c8aea7c nodeName:}" failed. No retries permitted until 2026-06-08 16:34:21.912947149 +0000 UTC m=+66.299135527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l9znh" (UniqueName: "kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh") pod "network-check-target-wtxrd" (UID: "c1c16413-d27a-4a43-a0c0-e40c1c8aea7c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 16:33:50.328280 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.328199 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="91930fe71b013ae46ebcb80591b87a7f689e97a0447819086ccddaa8331c30c7" exitCode=0 Jun 08 16:33:50.328280 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.328249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"91930fe71b013ae46ebcb80591b87a7f689e97a0447819086ccddaa8331c30c7"} Jun 08 16:33:50.902178 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.902094 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-92.ec2.internal" event="NodeReady" Jun 08 16:33:50.902347 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.902242 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 16:33:50.942108 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.942067 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:33:50.967056 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.967016 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx"] Jun 08 16:33:50.967227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.967168 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:50.970399 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.970166 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 16:33:50.970399 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.970211 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 16:33:50.970598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.970453 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 16:33:50.972101 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.972079 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w77h2\"" Jun 08 16:33:50.976030 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.976009 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 16:33:50.985050 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.985017 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-72zpv"] Jun 08 16:33:50.985216 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.985199 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:50.988302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.988277 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j69g9\"" Jun 08 16:33:50.988792 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.988772 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 08 16:33:50.988960 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:50.988804 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 08 16:33:51.009184 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.009147 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rbmgh"] Jun 08 16:33:51.009378 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.009318 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.012377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.012245 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 16:33:51.012377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.012311 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sqhpm\"" Jun 08 16:33:51.012377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.012351 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 16:33:51.012607 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.012245 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 16:33:51.036669 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.036626 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx"] Jun 08 16:33:51.036816 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.036682 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:33:51.036816 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.036703 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbmgh"] Jun 08 16:33:51.036816 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.036720 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-72zpv"] Jun 08 16:33:51.036816 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.036738 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.039887 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.039865 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 16:33:51.039998 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.039895 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cnvn6\"" Jun 08 16:33:51.039998 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.039874 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 16:33:51.112684 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.112625 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:33:51.112684 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.112674 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:33:51.112897 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.112675 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:33:51.116294 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116257 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 16:33:51.116443 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116320 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 16:33:51.116443 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116401 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 16:33:51.116611 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116481 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crf4q\"" Jun 08 16:33:51.116910 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116710 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mvqv5\"" Jun 08 16:33:51.116910 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.116743 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 16:33:51.120501 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6083b35-bb54-4772-b8ff-93f4bdbab53c-config-volume\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.120616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120517 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.120616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.120616 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120563 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120618 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120660 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120687 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120717 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6083b35-bb54-4772-b8ff-93f4bdbab53c-tmp-dir\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120768 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.120805 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6hk6\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120859 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgxsg\" (UniqueName: \"kubernetes.io/projected/a6083b35-bb54-4772-b8ff-93f4bdbab53c-kube-api-access-bgxsg\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120883 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/043c7ccf-bec6-49fb-ad22-e51a225045aa-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120901 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120927 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gkl6\" (UniqueName: \"kubernetes.io/projected/3b1c4918-0486-4c55-9efa-83db938280a8-kube-api-access-4gkl6\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120954 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b1c4918-0486-4c55-9efa-83db938280a8-cert\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.121096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.120979 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222071 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.221984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6083b35-bb54-4772-b8ff-93f4bdbab53c-tmp-dir\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.222071 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222035 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222071 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6hk6\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222337 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222109 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgxsg\" (UniqueName: \"kubernetes.io/projected/a6083b35-bb54-4772-b8ff-93f4bdbab53c-kube-api-access-bgxsg\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.222390 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/043c7ccf-bec6-49fb-ad22-e51a225045aa-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.222390 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222378 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222517 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222412 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6083b35-bb54-4772-b8ff-93f4bdbab53c-tmp-dir\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.222517 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gkl6\" (UniqueName: \"kubernetes.io/projected/3b1c4918-0486-4c55-9efa-83db938280a8-kube-api-access-4gkl6\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.222517 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222472 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b1c4918-0486-4c55-9efa-83db938280a8-cert\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.222517 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222498 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222543 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6083b35-bb54-4772-b8ff-93f4bdbab53c-config-volume\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222569 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222591 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222613 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222688 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222716 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.222811 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.222746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.223142 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.223090 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.223199 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223174 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:33:51.223248 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223238 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:33:51.723219998 +0000 UTC m=+36.109408370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:33:51.223311 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.223244 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/043c7ccf-bec6-49fb-ad22-e51a225045aa-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.223311 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.223272 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6083b35-bb54-4772-b8ff-93f4bdbab53c-config-volume\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.223311 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223293 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:33:51.223311 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223307 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:33:51.223523 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223359 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:33:51.723343877 +0000 UTC m=+36.109532253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:33:51.223523 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223418 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:33:51.223523 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.223450 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:51.723439649 +0000 UTC m=+36.109628030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:33:51.223855 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.223695 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.224228 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.224207 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.227550 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.227525 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.227671 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.227612 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.227723 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.227685 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3b1c4918-0486-4c55-9efa-83db938280a8-cert\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.232810 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.232784 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgxsg\" (UniqueName: \"kubernetes.io/projected/a6083b35-bb54-4772-b8ff-93f4bdbab53c-kube-api-access-bgxsg\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.232994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.232971 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.233149 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.233134 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6hk6\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.233321 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.233300 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gkl6\" (UniqueName: \"kubernetes.io/projected/3b1c4918-0486-4c55-9efa-83db938280a8-kube-api-access-4gkl6\") pod \"ingress-canary-72zpv\" (UID: \"3b1c4918-0486-4c55-9efa-83db938280a8\") " pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.320215 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.320177 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-72zpv" Jun 08 16:33:51.527598 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.527419 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-72zpv"] Jun 08 16:33:51.531378 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:33:51.531352 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b1c4918_0486_4c55_9efa_83db938280a8.slice/crio-347fec95f9464f8e906b89e652b2e3307af93dbc6a9e0b62dd559460f57b2c6d WatchSource:0}: Error finding container 347fec95f9464f8e906b89e652b2e3307af93dbc6a9e0b62dd559460f57b2c6d: Status 404 returned error can't find the container with id 347fec95f9464f8e906b89e652b2e3307af93dbc6a9e0b62dd559460f57b2c6d Jun 08 16:33:51.727138 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.727058 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:51.727138 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.727094 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:51.727138 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:51.727118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:51.727338 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727209 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:33:51.727338 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727271 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:52.727256896 +0000 UTC m=+37.113445268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:33:51.727338 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727209 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:33:51.727438 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727344 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:33:52.72733299 +0000 UTC m=+37.113521363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:33:51.727438 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727216 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:33:51.727438 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727360 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:33:51.727438 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:51.727380 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:33:52.727373931 +0000 UTC m=+37.113562302 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:33:52.334867 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:52.334826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-72zpv" event={"ID":"3b1c4918-0486-4c55-9efa-83db938280a8","Type":"ContainerStarted","Data":"347fec95f9464f8e906b89e652b2e3307af93dbc6a9e0b62dd559460f57b2c6d"} Jun 08 16:33:52.735620 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:52.735576 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:52.735620 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:52.735616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:52.735660 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735746 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735748 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735756 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735748 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735811 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:54.735791224 +0000 UTC m=+39.121979606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735828 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:33:54.735819104 +0000 UTC m=+39.122007483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:33:52.736090 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:52.735895 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:33:54.735889161 +0000 UTC m=+39.122077532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:33:54.753185 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:54.753083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:54.753185 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:54.753132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:54.753185 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:54.753156 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753258 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753264 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753264 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753329 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753332 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:33:58.753313671 +0000 UTC m=+43.139502043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753354 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:33:58.753343973 +0000 UTC m=+43.139532344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:33:54.753768 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:54.753369 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:33:58.753361171 +0000 UTC m=+43.139549548 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:33:55.409242 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.409130 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:55.501617 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.501578 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:55.601326 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.601286 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:55.702624 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.702535 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:55.803016 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.802983 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:55.903808 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:55.903770 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.003994 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.003910 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.103864 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.103819 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.547093 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.546986 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.608877 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.608844 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.708917 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.708885 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:56.809891 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:56.809797 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:57.107124 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.107089 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:57.207525 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.207496 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:57.353572 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.353535 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="c19f35032e3e1be21ab00617ac869211bb4c6d8937dc4864d788029218a730cc" exitCode=0 Jun 08 16:33:57.353572 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.353575 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"c19f35032e3e1be21ab00617ac869211bb4c6d8937dc4864d788029218a730cc"} Jun 08 16:33:57.355134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.355114 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-72zpv" event={"ID":"3b1c4918-0486-4c55-9efa-83db938280a8","Type":"ContainerStarted","Data":"73ba2cb333ac3c96e6822780ca15bed641c8c52d411886f34a37230092b790d8"} Jun 08 16:33:57.405334 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:57.405046 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-72zpv" podStartSLOduration=2.723109775 podStartE2EDuration="7.405022333s" totalImagesPullingTime="4.681912558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:57.404694432 +0000 UTC m=+41.790882826" watchObservedRunningTime="2026-06-08 16:33:57.405022333 +0000 UTC m=+41.791210728" Jun 08 16:33:58.360428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:58.360392 2559 generic.go:356] "Generic (PLEG): container finished" podID="4f962b4e-936c-4cac-a9f3-687fd8bd1c66" containerID="cc3f99f58c6ec98d1bdfc40fc8911856782c07afe6f2ccabe62d6d1751ebb751" exitCode=0 Jun 08 16:33:58.360832 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:58.360482 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerDied","Data":"cc3f99f58c6ec98d1bdfc40fc8911856782c07afe6f2ccabe62d6d1751ebb751"} Jun 08 16:33:58.788945 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:58.788834 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:33:58.788945 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:58.788898 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:33:58.788945 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:58.788931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789008 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789015 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789035 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789039 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789093 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:34:06.789073679 +0000 UTC m=+51.175262051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789105 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:34:06.789099578 +0000 UTC m=+51.175287949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:33:58.789219 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:33:58.789116 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:34:06.789111362 +0000 UTC m=+51.175299733 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:33:59.365727 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:59.365695 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" event={"ID":"4f962b4e-936c-4cac-a9f3-687fd8bd1c66","Type":"ContainerStarted","Data":"7c879af0fa1e88abd41e36715d3874e31fd76cba95d5d47f1b9d3f699dbf6fbf"} Jun 08 16:33:59.391728 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:59.391652 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6qkwf" podStartSLOduration=10.50208105 podStartE2EDuration="43.39161965s" totalImagesPullingTime="32.8895386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:33:59.390176124 +0000 UTC m=+43.776364519" watchObservedRunningTime="2026-06-08 16:33:59.39161965 +0000 UTC m=+43.777808044" Jun 08 16:33:59.524921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:59.524886 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:59.625151 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:59.625056 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:33:59.693713 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:33:59.693684 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:01.304327 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:01.304292 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:01.401115 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:01.401064 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:01.497232 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:01.497173 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:01.597169 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:01.597133 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:01.680802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:01.680769 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:05.071517 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:05.071487 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" limit=1024 Jun 08 16:34:06.850742 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:06.850701 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:34:06.850742 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:06.850745 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:06.850768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850867 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850872 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850884 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850878 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850970 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:34:22.850949371 +0000 UTC m=+67.237137749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.850995 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:34:22.850986779 +0000 UTC m=+67.237175152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:34:06.851241 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:06.851011 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:34:22.851003835 +0000 UTC m=+67.237192208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:34:17.934585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:17.934558 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tlf2q" Jun 08 16:34:21.866223 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.866184 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:34:21.866223 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.866232 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:34:21.868732 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.868702 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 16:34:21.868858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.868799 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 16:34:21.877458 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:21.877436 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 16:34:21.877575 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:21.877502 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:35:25.877485906 +0000 UTC m=+130.263674278 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : secret "metrics-daemon-secret" not found Jun 08 16:34:21.881317 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.881289 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/93ef7e77-f153-47bb-bd69-2a073996344d-original-pull-secret\") pod \"global-pull-secret-syncer-lq2k9\" (UID: \"93ef7e77-f153-47bb-bd69-2a073996344d\") " pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:34:21.967093 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.967049 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:34:21.970301 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.970283 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 16:34:21.981435 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.981412 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 16:34:21.990525 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:21.990491 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9znh\" (UniqueName: \"kubernetes.io/projected/c1c16413-d27a-4a43-a0c0-e40c1c8aea7c-kube-api-access-l9znh\") pod \"network-check-target-wtxrd\" (UID: \"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c\") " pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:34:22.026701 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.026665 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crf4q\"" Jun 08 16:34:22.034408 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.034380 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:34:22.036741 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.036711 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lq2k9" Jun 08 16:34:22.178971 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.178931 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wtxrd"] Jun 08 16:34:22.182291 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:34:22.182263 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1c16413_d27a_4a43_a0c0_e40c1c8aea7c.slice/crio-81a7eb2d377377ea53cd3782d85d6ac3b18be3101280442f1250d3b4dd332430 WatchSource:0}: Error finding container 81a7eb2d377377ea53cd3782d85d6ac3b18be3101280442f1250d3b4dd332430: Status 404 returned error can't find the container with id 81a7eb2d377377ea53cd3782d85d6ac3b18be3101280442f1250d3b4dd332430 Jun 08 16:34:22.193747 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.193724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lq2k9"] Jun 08 16:34:22.197598 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:34:22.197576 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ef7e77_f153_47bb_bd69_2a073996344d.slice/crio-91509ded0c602725a3eff03077850f4a1906ddec65a4efdadebaafd3e2e0bb03 WatchSource:0}: Error finding container 91509ded0c602725a3eff03077850f4a1906ddec65a4efdadebaafd3e2e0bb03: Status 404 returned error can't find the container with id 91509ded0c602725a3eff03077850f4a1906ddec65a4efdadebaafd3e2e0bb03 Jun 08 16:34:22.425335 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.425231 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wtxrd" event={"ID":"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c","Type":"ContainerStarted","Data":"81a7eb2d377377ea53cd3782d85d6ac3b18be3101280442f1250d3b4dd332430"} Jun 08 16:34:22.426174 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.426146 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lq2k9" event={"ID":"93ef7e77-f153-47bb-bd69-2a073996344d","Type":"ContainerStarted","Data":"91509ded0c602725a3eff03077850f4a1906ddec65a4efdadebaafd3e2e0bb03"} Jun 08 16:34:22.875548 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.875501 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:34:22.876044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.875556 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:34:22.876044 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:22.875592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:34:22.876044 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.875777 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:34:22.876044 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.875794 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:34:22.876044 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.875856 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:34:54.875835971 +0000 UTC m=+99.262024360 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:34:22.876308 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.876293 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:34:22.876374 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.876349 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:34:54.87633331 +0000 UTC m=+99.262521682 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:34:22.876436 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.876416 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:34:22.876491 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:22.876449 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:34:54.876437578 +0000 UTC m=+99.262625955 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:34:27.441174 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:27.441139 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lq2k9" event={"ID":"93ef7e77-f153-47bb-bd69-2a073996344d","Type":"ContainerStarted","Data":"6b84c54c8ce54189fe841d3d62deccb2c82a036f4df19d01488a77e396d5b80a"} Jun 08 16:34:27.442603 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:27.442578 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wtxrd" event={"ID":"c1c16413-d27a-4a43-a0c0-e40c1c8aea7c","Type":"ContainerStarted","Data":"ca45bf11479a254586b7d429fd38279f3e3bdb11e8cfca83e80fac8c9f55e5f3"} Jun 08 16:34:27.442751 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:27.442671 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:34:27.456772 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:27.456713 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lq2k9" podStartSLOduration=67.054490626 podStartE2EDuration="1m11.456698113s" totalImagesPullingTime="4.402207487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:34:27.456031552 +0000 UTC m=+71.842219947" watchObservedRunningTime="2026-06-08 16:34:27.456698113 +0000 UTC m=+71.842886506" Jun 08 16:34:27.472599 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:27.472539 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wtxrd" podStartSLOduration=67.059438052 podStartE2EDuration="1m11.472525695s" totalImagesPullingTime="4.413087643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:34:27.471802262 +0000 UTC m=+71.857990661" watchObservedRunningTime="2026-06-08 16:34:27.472525695 +0000 UTC m=+71.858714089" Jun 08 16:34:31.555966 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.555931 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb"] Jun 08 16:34:31.558745 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.558728 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.563286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.563241 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 08 16:34:31.563286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.563279 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 16:34:31.563515 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.563242 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-27x82\"" Jun 08 16:34:31.563515 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.563306 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 16:34:31.563515 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.563246 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 16:34:31.568756 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.568724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb"] Jun 08 16:34:31.639364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.639324 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7a73ecc-7093-43e2-8825-1f8809fce6e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.639364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.639372 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s2zj\" (UniqueName: \"kubernetes.io/projected/f7a73ecc-7093-43e2-8825-1f8809fce6e4-kube-api-access-8s2zj\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.739865 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.739831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7a73ecc-7093-43e2-8825-1f8809fce6e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.739974 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.739876 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s2zj\" (UniqueName: \"kubernetes.io/projected/f7a73ecc-7093-43e2-8825-1f8809fce6e4-kube-api-access-8s2zj\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.742528 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.742499 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f7a73ecc-7093-43e2-8825-1f8809fce6e4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.749041 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.749018 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s2zj\" (UniqueName: \"kubernetes.io/projected/f7a73ecc-7093-43e2-8825-1f8809fce6e4-kube-api-access-8s2zj\") pod \"managed-serviceaccount-addon-agent-6b6c648b6-9cwpb\" (UID: \"f7a73ecc-7093-43e2-8825-1f8809fce6e4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:31.875168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:31.875125 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" Jun 08 16:34:32.000970 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:32.000933 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb"] Jun 08 16:34:32.004660 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:34:32.004622 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7a73ecc_7093_43e2_8825_1f8809fce6e4.slice/crio-12839728f0e9a8c47d24dfbe03c82a11e93e7a2596d6558c160da6c2ccbe9591 WatchSource:0}: Error finding container 12839728f0e9a8c47d24dfbe03c82a11e93e7a2596d6558c160da6c2ccbe9591: Status 404 returned error can't find the container with id 12839728f0e9a8c47d24dfbe03c82a11e93e7a2596d6558c160da6c2ccbe9591 Jun 08 16:34:32.455895 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:32.455778 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" event={"ID":"f7a73ecc-7093-43e2-8825-1f8809fce6e4","Type":"ContainerStarted","Data":"12839728f0e9a8c47d24dfbe03c82a11e93e7a2596d6558c160da6c2ccbe9591"} Jun 08 16:34:35.466167 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:35.466128 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" event={"ID":"f7a73ecc-7093-43e2-8825-1f8809fce6e4","Type":"ContainerStarted","Data":"67a4e6e73a4811ae47e87423766cbc8ac9d247b09f1a78fd63a0f94576ce3f66"} Jun 08 16:34:35.481723 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:35.481673 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" podStartSLOduration=1.748701828 podStartE2EDuration="4.481659336s" totalImagesPullingTime="2.732957508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:34:35.481313275 +0000 UTC m=+79.867501670" watchObservedRunningTime="2026-06-08 16:34:35.481659336 +0000 UTC m=+79.867847724" Jun 08 16:34:54.912611 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:54.912557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:34:54.912611 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:54.912609 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:54.912663 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912735 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912786 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912801 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912822 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:35:58.912805012 +0000 UTC m=+163.298993384 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912735 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912868 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:35:58.91285263 +0000 UTC m=+163.299041019 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:34:54.913160 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:34:54.912888 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:35:58.912877301 +0000 UTC m=+163.299065675 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:34:58.448800 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:34:58.448767 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wtxrd" Jun 08 16:35:25.942112 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:25.942059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:35:25.942626 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:25.942218 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 16:35:25.942626 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:25.942289 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:37:27.942272563 +0000 UTC m=+252.328460935 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : secret "metrics-daemon-secret" not found Jun 08 16:35:53.980857 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:53.980808 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-555685f464-v6hcx" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" Jun 08 16:35:53.996108 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:53.996080 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" podUID="043c7ccf-bec6-49fb-ad22-e51a225045aa" Jun 08 16:35:54.046645 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:54.046596 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rbmgh" podUID="a6083b35-bb54-4772-b8ff-93f4bdbab53c" Jun 08 16:35:54.129965 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:54.129919 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:35:54.654146 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:54.654111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:35:54.654333 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:54.654110 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:35:54.654333 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:54.654111 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:35:58.987820 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:58.987778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:35:58.987820 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:58.987820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:35:58.987842 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.987937 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.987970 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.987945 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.988023 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-555685f464-v6hcx: secret "image-registry-tls" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.988013 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert podName:043c7ccf-bec6-49fb-ad22-e51a225045aa nodeName:}" failed. No retries permitted until 2026-06-08 16:38:00.987997069 +0000 UTC m=+285.374185440 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-btgmx" (UID: "043c7ccf-bec6-49fb-ad22-e51a225045aa") : secret "networking-console-plugin-cert" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.988067 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls podName:a6083b35-bb54-4772-b8ff-93f4bdbab53c nodeName:}" failed. No retries permitted until 2026-06-08 16:38:00.988054102 +0000 UTC m=+285.374242476 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls") pod "dns-default-rbmgh" (UID: "a6083b35-bb54-4772-b8ff-93f4bdbab53c") : secret "dns-default-metrics-tls" not found Jun 08 16:35:58.988333 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:35:58.988077 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls podName:d4a7ce8d-5e63-40db-bc8b-a94ce59753fe nodeName:}" failed. No retries permitted until 2026-06-08 16:38:00.988071749 +0000 UTC m=+285.374260121 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls") pod "image-registry-555685f464-v6hcx" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe") : secret "image-registry-tls" not found Jun 08 16:36:06.113381 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:36:06.113281 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:36:35.751360 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:36:35.751328 2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a73ecc-7093-43e2-8825-1f8809fce6e4" containerID="67a4e6e73a4811ae47e87423766cbc8ac9d247b09f1a78fd63a0f94576ce3f66" exitCode=255 Jun 08 16:36:35.751795 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:36:35.751401 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" event={"ID":"f7a73ecc-7093-43e2-8825-1f8809fce6e4","Type":"ContainerDied","Data":"67a4e6e73a4811ae47e87423766cbc8ac9d247b09f1a78fd63a0f94576ce3f66"} Jun 08 16:36:35.756893 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:36:35.756869 2559 scope.go:117] "RemoveContainer" containerID="67a4e6e73a4811ae47e87423766cbc8ac9d247b09f1a78fd63a0f94576ce3f66" Jun 08 16:36:36.756786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:36:36.756750 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6b6c648b6-9cwpb" event={"ID":"f7a73ecc-7093-43e2-8825-1f8809fce6e4","Type":"ContainerStarted","Data":"195608e1bd41e96063b1fc52e55cabc41143f104f5ac4f68dfbe93500936497d"} Jun 08 16:37:24.000941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.000905 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-69c5b8477c-6djtn"] Jun 08 16:37:24.003858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.003841 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.006697 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.006671 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-t9dwg\"" Jun 08 16:37:24.006843 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.006823 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 08 16:37:24.006903 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.006824 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 16:37:24.007123 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.007105 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 08 16:37:24.007195 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.007136 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 08 16:37:24.007489 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.007474 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 16:37:24.007867 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.007853 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 08 16:37:24.018172 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.018144 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69c5b8477c-6djtn"] Jun 08 16:37:24.082028 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.081990 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.082216 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.082042 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-stats-auth\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.082216 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.082072 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qsst\" (UniqueName: \"kubernetes.io/projected/bc858a75-84d7-4e35-b231-bdc15ef0624f-kube-api-access-2qsst\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.082317 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.082211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-default-certificate\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.082317 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.082237 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183151 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.183109 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-default-certificate\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183151 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.183150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.183170 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.183201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-stats-auth\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.183223 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qsst\" (UniqueName: \"kubernetes.io/projected/bc858a75-84d7-4e35-b231-bdc15ef0624f-kube-api-access-2qsst\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.183280 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.183339 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:24.683317479 +0000 UTC m=+249.069505851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:24.183421 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.183366 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:24.683356134 +0000 UTC m=+249.069544506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:24.185769 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.185751 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-default-certificate\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.185844 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.185781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-stats-auth\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.192655 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.192604 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qsst\" (UniqueName: \"kubernetes.io/projected/bc858a75-84d7-4e35-b231-bdc15ef0624f-kube-api-access-2qsst\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.687237 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.687194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.687237 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:24.687236 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:24.687464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.687351 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:24.687464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.687377 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:25.6873581 +0000 UTC m=+250.073546489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:24.687464 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:24.687399 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:25.687388086 +0000 UTC m=+250.073576475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:25.695029 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:25.694971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:25.695479 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:25.695089 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:25.695479 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:25.695157 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:25.695479 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:25.695200 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:27.695186207 +0000 UTC m=+252.081374581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:25.695479 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:25.695227 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:27.695209538 +0000 UTC m=+252.081397925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:27.711324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:27.711274 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:27.711324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:27.711322 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:27.711826 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:27.711442 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:27.711826 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:27.711504 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:31.711489429 +0000 UTC m=+256.097677803 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:27.711826 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:27.711517 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:31.711511762 +0000 UTC m=+256.097700134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:28.013895 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:28.013808 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:37:28.014037 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:28.013953 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 16:37:28.014037 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:28.014031 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs podName:f972084f-acf6-4427-b7ea-0f340e15343b nodeName:}" failed. No retries permitted until 2026-06-08 16:39:30.013999397 +0000 UTC m=+374.400187768 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs") pod "network-metrics-daemon-5z57p" (UID: "f972084f-acf6-4427-b7ea-0f340e15343b") : secret "metrics-daemon-secret" not found Jun 08 16:37:31.298910 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:31.298881 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kxw8p_1dedd458-09c2-40f1-9dfa-2bd02e1f8965/dns-node-resolver/0.log" Jun 08 16:37:31.740465 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:31.740420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:31.740698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:31.740471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:31.740698 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:31.740601 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:31.740698 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:31.740604 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:39.740582472 +0000 UTC m=+264.126770844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:31.740698 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:31.740696 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:39.740676716 +0000 UTC m=+264.126865092 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:32.099853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.099823 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bd2x9_40a14650-8507-4ce8-8301-987731013a88/node-ca/0.log" Jun 08 16:37:32.431571 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.431482 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-tnf85"] Jun 08 16:37:32.434400 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.434344 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.437170 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.437146 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vklmx\"" Jun 08 16:37:32.437286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.437146 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 08 16:37:32.438335 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.438311 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 08 16:37:32.438335 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.438324 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 08 16:37:32.438472 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.438324 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 08 16:37:32.445297 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.445272 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwf4b\" (UniqueName: \"kubernetes.io/projected/73cfa640-94d6-40b4-a309-289fadea35d7-kube-api-access-cwf4b\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.445406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.445359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73cfa640-94d6-40b4-a309-289fadea35d7-signing-cabundle\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.445406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.445399 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73cfa640-94d6-40b4-a309-289fadea35d7-signing-key\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.449814 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.449785 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-tnf85"] Jun 08 16:37:32.546376 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.546337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73cfa640-94d6-40b4-a309-289fadea35d7-signing-key\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.546590 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.546430 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwf4b\" (UniqueName: \"kubernetes.io/projected/73cfa640-94d6-40b4-a309-289fadea35d7-kube-api-access-cwf4b\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.546590 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.546473 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73cfa640-94d6-40b4-a309-289fadea35d7-signing-cabundle\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.547134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.547114 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73cfa640-94d6-40b4-a309-289fadea35d7-signing-cabundle\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.548970 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.548943 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73cfa640-94d6-40b4-a309-289fadea35d7-signing-key\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.555827 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.555804 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwf4b\" (UniqueName: \"kubernetes.io/projected/73cfa640-94d6-40b4-a309-289fadea35d7-kube-api-access-cwf4b\") pod \"service-ca-7d74d578b8-tnf85\" (UID: \"73cfa640-94d6-40b4-a309-289fadea35d7\") " pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.743488 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.743396 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" Jun 08 16:37:32.869708 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.869675 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-tnf85"] Jun 08 16:37:32.872349 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:37:32.872322 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73cfa640_94d6_40b4_a309_289fadea35d7.slice/crio-ecdb76657de1a6fa57ab74547f2651be26f7c39396f558aebe27777d661fa74e WatchSource:0}: Error finding container ecdb76657de1a6fa57ab74547f2651be26f7c39396f558aebe27777d661fa74e: Status 404 returned error can't find the container with id ecdb76657de1a6fa57ab74547f2651be26f7c39396f558aebe27777d661fa74e Jun 08 16:37:32.903169 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.903136 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-72zpv_3b1c4918-0486-4c55-9efa-83db938280a8/serve-healthcheck-canary/0.log" Jun 08 16:37:32.925412 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:32.925375 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" event={"ID":"73cfa640-94d6-40b4-a309-289fadea35d7","Type":"ContainerStarted","Data":"ecdb76657de1a6fa57ab74547f2651be26f7c39396f558aebe27777d661fa74e"} Jun 08 16:37:35.939227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:35.938409 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" event={"ID":"73cfa640-94d6-40b4-a309-289fadea35d7","Type":"ContainerStarted","Data":"b8193dcd9851a6d24e8d761235c6237330e4af0de7d46d6ac9149959ca3644a0"} Jun 08 16:37:35.956711 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:35.956627 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-tnf85" podStartSLOduration=1.944033207 podStartE2EDuration="3.95660769s" totalImagesPullingTime="2.012574483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:37:35.955878606 +0000 UTC m=+260.342067003" watchObservedRunningTime="2026-06-08 16:37:35.95660769 +0000 UTC m=+260.342796084" Jun 08 16:37:39.808239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:39.808192 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:39.809021 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:39.808995 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:39.809284 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:39.809268 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:55.809246403 +0000 UTC m=+280.195434788 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : configmap references non-existent config key: service-ca.crt Jun 08 16:37:39.809933 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:39.809912 2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 16:37:39.810101 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:39.810090 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs podName:bc858a75-84d7-4e35-b231-bdc15ef0624f nodeName:}" failed. No retries permitted until 2026-06-08 16:37:55.81006467 +0000 UTC m=+280.196253057 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs") pod "router-default-69c5b8477c-6djtn" (UID: "bc858a75-84d7-4e35-b231-bdc15ef0624f") : secret "router-metrics-certs-default" not found Jun 08 16:37:55.840848 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:55.840790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:55.840848 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:55.840852 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:55.841428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:55.841393 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc858a75-84d7-4e35-b231-bdc15ef0624f-service-ca-bundle\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:55.843340 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:55.843319 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc858a75-84d7-4e35-b231-bdc15ef0624f-metrics-certs\") pod \"router-default-69c5b8477c-6djtn\" (UID: \"bc858a75-84d7-4e35-b231-bdc15ef0624f\") " pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:56.112725 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:56.112610 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:56.253493 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:56.250564 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-69c5b8477c-6djtn"] Jun 08 16:37:56.254788 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:37:56.254760 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc858a75_84d7_4e35_b231_bdc15ef0624f.slice/crio-f2aad5e1d73da47c6a7dddefc3fb4e7bf8d1cf31378069acdda1a51f4f49e16f WatchSource:0}: Error finding container f2aad5e1d73da47c6a7dddefc3fb4e7bf8d1cf31378069acdda1a51f4f49e16f: Status 404 returned error can't find the container with id f2aad5e1d73da47c6a7dddefc3fb4e7bf8d1cf31378069acdda1a51f4f49e16f Jun 08 16:37:57.000666 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:57.000621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69c5b8477c-6djtn" event={"ID":"bc858a75-84d7-4e35-b231-bdc15ef0624f","Type":"ContainerStarted","Data":"89eef507445d24812cc7ef1dbaf425482e676b0e127e11fbc202e39b9d8852d0"} Jun 08 16:37:57.001036 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:57.000679 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-69c5b8477c-6djtn" event={"ID":"bc858a75-84d7-4e35-b231-bdc15ef0624f","Type":"ContainerStarted","Data":"f2aad5e1d73da47c6a7dddefc3fb4e7bf8d1cf31378069acdda1a51f4f49e16f"} Jun 08 16:37:57.113405 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:57.113365 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:57.115921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:57.115897 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:57.135831 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:57.135776 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-69c5b8477c-6djtn" podStartSLOduration=34.135757827 podStartE2EDuration="34.135757827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:37:57.020813425 +0000 UTC m=+281.407001820" watchObservedRunningTime="2026-06-08 16:37:57.135757827 +0000 UTC m=+281.521946221" Jun 08 16:37:57.654515 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:57.654464 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" podUID="043c7ccf-bec6-49fb-ad22-e51a225045aa" Jun 08 16:37:57.654698 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:57.654619 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rbmgh" podUID="a6083b35-bb54-4772-b8ff-93f4bdbab53c" Jun 08 16:37:57.654764 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:37:57.654707 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-555685f464-v6hcx" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" Jun 08 16:37:58.003324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:58.003236 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:37:58.003324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:58.003249 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:37:58.003324 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:58.003263 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:37:58.003823 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:58.003503 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:37:58.004884 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:37:58.004863 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-69c5b8477c-6djtn" Jun 08 16:38:00.664821 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.664789 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cc9qq"] Jun 08 16:38:00.668147 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.668130 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.671906 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.671882 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 16:38:00.671906 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.671893 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 16:38:00.673148 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.673129 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 16:38:00.673148 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.673143 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 16:38:00.673288 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.673201 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4chrf\"" Jun 08 16:38:00.699865 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.699833 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cc9qq"] Jun 08 16:38:00.780122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.780087 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee1356b5-bda6-47fb-94d6-e6960d12c53f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.780307 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.780142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f59zl\" (UniqueName: \"kubernetes.io/projected/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-api-access-f59zl\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.780307 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.780211 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.780307 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.780252 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee1356b5-bda6-47fb-94d6-e6960d12c53f-data-volume\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.780406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.780345 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee1356b5-bda6-47fb-94d6-e6960d12c53f-crio-socket\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.880853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880753 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee1356b5-bda6-47fb-94d6-e6960d12c53f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.880853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f59zl\" (UniqueName: \"kubernetes.io/projected/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-api-access-f59zl\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.880853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880820 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.880853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880850 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee1356b5-bda6-47fb-94d6-e6960d12c53f-data-volume\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.881184 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880905 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee1356b5-bda6-47fb-94d6-e6960d12c53f-crio-socket\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.881184 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.880988 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee1356b5-bda6-47fb-94d6-e6960d12c53f-crio-socket\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.881269 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.881254 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee1356b5-bda6-47fb-94d6-e6960d12c53f-data-volume\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.881482 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.881461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.883348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.883327 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee1356b5-bda6-47fb-94d6-e6960d12c53f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.890509 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.890476 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f59zl\" (UniqueName: \"kubernetes.io/projected/ee1356b5-bda6-47fb-94d6-e6960d12c53f-kube-api-access-f59zl\") pod \"insights-runtime-extractor-cc9qq\" (UID: \"ee1356b5-bda6-47fb-94d6-e6960d12c53f\") " pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:00.977869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:00.977833 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cc9qq" Jun 08 16:38:01.083115 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.083081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:38:01.083280 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.083123 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:38:01.083280 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.083193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:01.085662 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.085611 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"image-registry-555685f464-v6hcx\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:01.085794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.085609 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6083b35-bb54-4772-b8ff-93f4bdbab53c-metrics-tls\") pod \"dns-default-rbmgh\" (UID: \"a6083b35-bb54-4772-b8ff-93f4bdbab53c\") " pod="openshift-dns/dns-default-rbmgh" Jun 08 16:38:01.085794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.085759 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/043c7ccf-bec6-49fb-ad22-e51a225045aa-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-btgmx\" (UID: \"043c7ccf-bec6-49fb-ad22-e51a225045aa\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:38:01.113337 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.113294 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cc9qq"] Jun 08 16:38:01.115957 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:01.115926 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1356b5_bda6_47fb_94d6_e6960d12c53f.slice/crio-64b718652d3eaebf004df8d3b86bdb9c73b146dbea42187eb790aa0c1787bff3 WatchSource:0}: Error finding container 64b718652d3eaebf004df8d3b86bdb9c73b146dbea42187eb790aa0c1787bff3: Status 404 returned error can't find the container with id 64b718652d3eaebf004df8d3b86bdb9c73b146dbea42187eb790aa0c1787bff3 Jun 08 16:38:01.306853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.306813 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-w77h2\"" Jun 08 16:38:01.306853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.306834 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j69g9\"" Jun 08 16:38:01.306853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.306859 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cnvn6\"" Jun 08 16:38:01.314331 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.314293 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:01.314331 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.314311 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:38:01.314505 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.314348 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" Jun 08 16:38:01.471020 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.469054 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:38:01.473686 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:01.472246 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a7ce8d_5e63_40db_bc8b_a94ce59753fe.slice/crio-418715ea4a5f9e088676727d8f5446ac5014b3d85bcfd9e6a027c29b0571311d WatchSource:0}: Error finding container 418715ea4a5f9e088676727d8f5446ac5014b3d85bcfd9e6a027c29b0571311d: Status 404 returned error can't find the container with id 418715ea4a5f9e088676727d8f5446ac5014b3d85bcfd9e6a027c29b0571311d Jun 08 16:38:01.688379 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.688277 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx"] Jun 08 16:38:01.691168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:01.691114 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rbmgh"] Jun 08 16:38:01.730834 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:01.730798 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043c7ccf_bec6_49fb_ad22_e51a225045aa.slice/crio-2bcb8da7b713da7ef194b7729efa3cf34cd6b48bfdb75dcc1573d1d6c58de663 WatchSource:0}: Error finding container 2bcb8da7b713da7ef194b7729efa3cf34cd6b48bfdb75dcc1573d1d6c58de663: Status 404 returned error can't find the container with id 2bcb8da7b713da7ef194b7729efa3cf34cd6b48bfdb75dcc1573d1d6c58de663 Jun 08 16:38:01.731427 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:01.731410 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6083b35_bb54_4772_b8ff_93f4bdbab53c.slice/crio-b40ae851ebfa0fb2f6ba067d67634c998f091276ba92ef8e58b6cb346bb0ea51 WatchSource:0}: Error finding container b40ae851ebfa0fb2f6ba067d67634c998f091276ba92ef8e58b6cb346bb0ea51: Status 404 returned error can't find the container with id b40ae851ebfa0fb2f6ba067d67634c998f091276ba92ef8e58b6cb346bb0ea51 Jun 08 16:38:02.015925 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.015827 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmgh" event={"ID":"a6083b35-bb54-4772-b8ff-93f4bdbab53c","Type":"ContainerStarted","Data":"b40ae851ebfa0fb2f6ba067d67634c998f091276ba92ef8e58b6cb346bb0ea51"} Jun 08 16:38:02.016910 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.016885 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" event={"ID":"043c7ccf-bec6-49fb-ad22-e51a225045aa","Type":"ContainerStarted","Data":"2bcb8da7b713da7ef194b7729efa3cf34cd6b48bfdb75dcc1573d1d6c58de663"} Jun 08 16:38:02.018289 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.018270 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555685f464-v6hcx" event={"ID":"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe","Type":"ContainerStarted","Data":"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2"} Jun 08 16:38:02.018364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.018302 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555685f464-v6hcx" event={"ID":"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe","Type":"ContainerStarted","Data":"418715ea4a5f9e088676727d8f5446ac5014b3d85bcfd9e6a027c29b0571311d"} Jun 08 16:38:02.018422 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.018378 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:02.019905 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.019886 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc9qq" event={"ID":"ee1356b5-bda6-47fb-94d6-e6960d12c53f","Type":"ContainerStarted","Data":"3e170f65bcaf2093063f879eb0317c838adc68f8f327d13dd4cf5bd30ca0b15e"} Jun 08 16:38:02.019979 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.019924 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc9qq" event={"ID":"ee1356b5-bda6-47fb-94d6-e6960d12c53f","Type":"ContainerStarted","Data":"efcdfdd81fb3fddcfc987e0149ca1b7cd71e42e7695da91bec74906663cee22f"} Jun 08 16:38:02.019979 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.019941 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc9qq" event={"ID":"ee1356b5-bda6-47fb-94d6-e6960d12c53f","Type":"ContainerStarted","Data":"64b718652d3eaebf004df8d3b86bdb9c73b146dbea42187eb790aa0c1787bff3"} Jun 08 16:38:02.052959 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:02.052900 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-555685f464-v6hcx" podStartSLOduration=286.052884313 podStartE2EDuration="4m46.052884313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:02.052620217 +0000 UTC m=+286.438808612" watchObservedRunningTime="2026-06-08 16:38:02.052884313 +0000 UTC m=+286.439072707" Jun 08 16:38:03.025387 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.025291 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" event={"ID":"043c7ccf-bec6-49fb-ad22-e51a225045aa","Type":"ContainerStarted","Data":"6c0a1f7bce4f733bbf10ddf237c660bf077adf317d91352a2506b6e040c9acac"} Jun 08 16:38:03.044247 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.044193 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-btgmx" podStartSLOduration=284.078948021 podStartE2EDuration="4m45.044173194s" totalImagesPullingTime="965.225173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:03.043866308 +0000 UTC m=+287.430054703" watchObservedRunningTime="2026-06-08 16:38:03.044173194 +0000 UTC m=+287.430361589" Jun 08 16:38:03.253821 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.253758 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5"] Jun 08 16:38:03.258765 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.258725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.263686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.263656 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 16:38:03.263873 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.263855 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 08 16:38:03.264083 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.264066 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8sh2t\"" Jun 08 16:38:03.264174 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.264131 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 08 16:38:03.264254 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.264235 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 16:38:03.264354 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.264293 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 16:38:03.269078 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.269009 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5"] Jun 08 16:38:03.402618 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.402580 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcc38e97-2f6a-4dbe-baa6-5429bee97124-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.402836 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.402622 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhqhk\" (UniqueName: \"kubernetes.io/projected/dcc38e97-2f6a-4dbe-baa6-5429bee97124-kube-api-access-zhqhk\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.402836 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.402740 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.402956 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.402857 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.504123 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.504087 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcc38e97-2f6a-4dbe-baa6-5429bee97124-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.504123 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.504125 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhqhk\" (UniqueName: \"kubernetes.io/projected/dcc38e97-2f6a-4dbe-baa6-5429bee97124-kube-api-access-zhqhk\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.504381 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.504352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.504485 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.504447 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.504940 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.504883 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dcc38e97-2f6a-4dbe-baa6-5429bee97124-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.507691 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.507665 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.507934 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.507911 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dcc38e97-2f6a-4dbe-baa6-5429bee97124-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.515052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.515022 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhqhk\" (UniqueName: \"kubernetes.io/projected/dcc38e97-2f6a-4dbe-baa6-5429bee97124-kube-api-access-zhqhk\") pod \"prometheus-operator-7f7d445d84-9dgf5\" (UID: \"dcc38e97-2f6a-4dbe-baa6-5429bee97124\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:03.572513 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:03.572476 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" Jun 08 16:38:04.091236 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:04.091200 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5"] Jun 08 16:38:04.097600 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:04.097567 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc38e97_2f6a_4dbe_baa6_5429bee97124.slice/crio-147dc187ae82b19840b77841862cce92d1bdee228efe7a0dbe806ed119fc2ff7 WatchSource:0}: Error finding container 147dc187ae82b19840b77841862cce92d1bdee228efe7a0dbe806ed119fc2ff7: Status 404 returned error can't find the container with id 147dc187ae82b19840b77841862cce92d1bdee228efe7a0dbe806ed119fc2ff7 Jun 08 16:38:05.034327 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.034271 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmgh" event={"ID":"a6083b35-bb54-4772-b8ff-93f4bdbab53c","Type":"ContainerStarted","Data":"a7fee157950f9e0a50e319ed2ea0dd0470047b690fcf0252302d9440ba4dbfee"} Jun 08 16:38:05.034526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.034394 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rbmgh" event={"ID":"a6083b35-bb54-4772-b8ff-93f4bdbab53c","Type":"ContainerStarted","Data":"2bb18fb0223d83c66bcc315bbef23673b866af672519e3cefa45413cbadd2287"} Jun 08 16:38:05.034526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.034428 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:38:05.035521 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.035490 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" event={"ID":"dcc38e97-2f6a-4dbe-baa6-5429bee97124","Type":"ContainerStarted","Data":"147dc187ae82b19840b77841862cce92d1bdee228efe7a0dbe806ed119fc2ff7"} Jun 08 16:38:05.037473 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.037449 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cc9qq" event={"ID":"ee1356b5-bda6-47fb-94d6-e6960d12c53f","Type":"ContainerStarted","Data":"96eaabac858ea62679ccf8aaddd7626927f97d988700fae052ac757b6b0f61ca"} Jun 08 16:38:05.055527 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.055469 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rbmgh" podStartSLOduration=252.83102883 podStartE2EDuration="4m15.055453495s" totalImagesPullingTime="2.224424665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:05.053550619 +0000 UTC m=+289.439739013" watchObservedRunningTime="2026-06-08 16:38:05.055453495 +0000 UTC m=+289.441641917" Jun 08 16:38:05.072085 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:05.072030 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cc9qq" podStartSLOduration=2.376124511 podStartE2EDuration="5.072014611s" totalImagesPullingTime="2.6958901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 16:38:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:05.071590438 +0000 UTC m=+289.457778832" watchObservedRunningTime="2026-06-08 16:38:05.072014611 +0000 UTC m=+289.458203005" Jun 08 16:38:06.042560 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:06.042519 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" event={"ID":"dcc38e97-2f6a-4dbe-baa6-5429bee97124","Type":"ContainerStarted","Data":"b003f63153dab8d12b7253dc8484667cf22a4adad5fcbea687b7f0138f27512a"} Jun 08 16:38:06.043057 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:06.042583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" event={"ID":"dcc38e97-2f6a-4dbe-baa6-5429bee97124","Type":"ContainerStarted","Data":"ea71db950d24e721dd2407a3d6b09a4e690b1377ab0ff178f843471a67328a92"} Jun 08 16:38:06.060091 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:06.060040 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9dgf5" podStartSLOduration=1.5922484300000002 podStartE2EDuration="3.060022227s" totalImagesPullingTime="1.467773797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:38:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:06.059139542 +0000 UTC m=+290.445327941" watchObservedRunningTime="2026-06-08 16:38:06.060022227 +0000 UTC m=+290.446210682" Jun 08 16:38:07.616436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.616395 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8"] Jun 08 16:38:07.621941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.621912 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.624872 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.624846 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 08 16:38:07.625074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.625056 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6bj9b\"" Jun 08 16:38:07.625144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.624908 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 08 16:38:07.634961 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.634425 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8"] Jun 08 16:38:07.665988 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.665949 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-scwpv"] Jun 08 16:38:07.673336 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.673069 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.676978 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.676923 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 16:38:07.677168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.676968 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qxfrb\"" Jun 08 16:38:07.677233 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.676976 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 16:38:07.677284 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.677111 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 16:38:07.736968 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.736930 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.736968 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.736969 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.737179 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.736994 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5t9t\" (UniqueName: \"kubernetes.io/projected/2938d7ea-6002-4e59-bcb3-eb82515bf947-kube-api-access-d5t9t\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.737179 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.737027 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2938d7ea-6002-4e59-bcb3-eb82515bf947-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.838429 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838384 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.838429 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838425 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5t9t\" (UniqueName: \"kubernetes.io/projected/2938d7ea-6002-4e59-bcb3-eb82515bf947-kube-api-access-d5t9t\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.838720 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838459 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-accelerators-collector-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838720 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838559 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2938d7ea-6002-4e59-bcb3-eb82515bf947-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.838720 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838604 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-wtmp\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838720 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-root\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838927 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838730 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838927 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838758 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2hl\" (UniqueName: \"kubernetes.io/projected/bc925f5a-6875-4f48-8be6-e2eca7639413-kube-api-access-fw2hl\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838927 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838831 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-sys\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838927 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838865 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-metrics-client-ca\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.838927 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838891 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-tls\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.839123 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838941 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-textfile\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.839123 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.838981 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.839544 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.839516 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2938d7ea-6002-4e59-bcb3-eb82515bf947-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.841312 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.841281 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.841447 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.841427 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2938d7ea-6002-4e59-bcb3-eb82515bf947-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.847619 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.847596 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5t9t\" (UniqueName: \"kubernetes.io/projected/2938d7ea-6002-4e59-bcb3-eb82515bf947-kube-api-access-d5t9t\") pod \"openshift-state-metrics-65f78d5c66-dv2l8\" (UID: \"2938d7ea-6002-4e59-bcb3-eb82515bf947\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.935801 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.935698 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" Jun 08 16:38:07.940296 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940275 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-textfile\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940388 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940314 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-accelerators-collector-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940388 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940334 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-wtmp\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940388 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940379 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-root\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940401 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940418 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2hl\" (UniqueName: \"kubernetes.io/projected/bc925f5a-6875-4f48-8be6-e2eca7639413-kube-api-access-fw2hl\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940657 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940577 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-wtmp\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940657 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940610 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-root\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940683 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-sys\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940715 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bc925f5a-6875-4f48-8be6-e2eca7639413-sys\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940716 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-textfile\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940752 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-metrics-client-ca\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940952 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940776 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-tls\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.940952 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.940905 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-accelerators-collector-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.941226 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.941209 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bc925f5a-6875-4f48-8be6-e2eca7639413-metrics-client-ca\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.943100 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.943064 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.943671 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.943625 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bc925f5a-6875-4f48-8be6-e2eca7639413-node-exporter-tls\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.956739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.956708 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2hl\" (UniqueName: \"kubernetes.io/projected/bc925f5a-6875-4f48-8be6-e2eca7639413-kube-api-access-fw2hl\") pod \"node-exporter-scwpv\" (UID: \"bc925f5a-6875-4f48-8be6-e2eca7639413\") " pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:07.985785 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:07.985749 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-scwpv" Jun 08 16:38:08.001732 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:08.001695 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc925f5a_6875_4f48_8be6_e2eca7639413.slice/crio-76b1eb0efa2250fbd98c65fc3bf24c80267f0b5aed9d2f37541ed9d23f170be2 WatchSource:0}: Error finding container 76b1eb0efa2250fbd98c65fc3bf24c80267f0b5aed9d2f37541ed9d23f170be2: Status 404 returned error can't find the container with id 76b1eb0efa2250fbd98c65fc3bf24c80267f0b5aed9d2f37541ed9d23f170be2 Jun 08 16:38:08.052627 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:08.052555 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-scwpv" event={"ID":"bc925f5a-6875-4f48-8be6-e2eca7639413","Type":"ContainerStarted","Data":"76b1eb0efa2250fbd98c65fc3bf24c80267f0b5aed9d2f37541ed9d23f170be2"} Jun 08 16:38:08.090672 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:08.090611 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8"] Jun 08 16:38:08.093973 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:08.093948 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2938d7ea_6002_4e59_bcb3_eb82515bf947.slice/crio-bf54e76a9e71e3c5153fcc2faccdfd49ee65b7b21281477ed6cf7ac7121a604a WatchSource:0}: Error finding container bf54e76a9e71e3c5153fcc2faccdfd49ee65b7b21281477ed6cf7ac7121a604a: Status 404 returned error can't find the container with id bf54e76a9e71e3c5153fcc2faccdfd49ee65b7b21281477ed6cf7ac7121a604a Jun 08 16:38:09.061122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:09.061082 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" event={"ID":"2938d7ea-6002-4e59-bcb3-eb82515bf947","Type":"ContainerStarted","Data":"2df60507aa5e7d6eaf65a0e9068bebf478d6b2af4d6c5525551aef7a4767295a"} Jun 08 16:38:09.061562 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:09.061152 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" event={"ID":"2938d7ea-6002-4e59-bcb3-eb82515bf947","Type":"ContainerStarted","Data":"e625253211112ec1c6c04e32858df6c0248782084c20be861e31cc5a9197cbf5"} Jun 08 16:38:09.061562 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:09.061182 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" event={"ID":"2938d7ea-6002-4e59-bcb3-eb82515bf947","Type":"ContainerStarted","Data":"bf54e76a9e71e3c5153fcc2faccdfd49ee65b7b21281477ed6cf7ac7121a604a"} Jun 08 16:38:09.114725 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:38:09.114683 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5z57p" podUID="f972084f-acf6-4427-b7ea-0f340e15343b" Jun 08 16:38:10.066340 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.066307 2559 generic.go:356] "Generic (PLEG): container finished" podID="bc925f5a-6875-4f48-8be6-e2eca7639413" containerID="05aa2003fded5947342bae23429e7521ba5c055fc44704e7fb100ca450030c1a" exitCode=0 Jun 08 16:38:10.066836 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.066404 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-scwpv" event={"ID":"bc925f5a-6875-4f48-8be6-e2eca7639413","Type":"ContainerDied","Data":"05aa2003fded5947342bae23429e7521ba5c055fc44704e7fb100ca450030c1a"} Jun 08 16:38:10.068624 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.068604 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" event={"ID":"2938d7ea-6002-4e59-bcb3-eb82515bf947","Type":"ContainerStarted","Data":"12c50119d5956481ff7efa0aa56b17a6565de85ca567580b1448262ffa69c462"} Jun 08 16:38:10.109895 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.109843 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dv2l8" podStartSLOduration=2.190970285 podStartE2EDuration="3.109823551s" totalImagesPullingTime="918.853266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:10.109247622 +0000 UTC m=+294.495436017" watchObservedRunningTime="2026-06-08 16:38:10.109823551 +0000 UTC m=+294.496011944" Jun 08 16:38:10.735508 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.735473 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d67f47b99-pbm6b"] Jun 08 16:38:10.738790 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.738766 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.741802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741718 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 08 16:38:10.741947 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741829 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mpp2d\"" Jun 08 16:38:10.741947 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741886 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 08 16:38:10.741947 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741893 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4nte6j0qj6lup\"" Jun 08 16:38:10.741947 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741832 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 08 16:38:10.742136 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.741832 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 08 16:38:10.742136 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.742067 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 08 16:38:10.753347 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.753307 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d67f47b99-pbm6b"] Jun 08 16:38:10.864230 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864424 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864246 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864424 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864314 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864424 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-metrics-client-ca\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864424 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864620 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864440 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-grpc-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864620 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864478 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.864620 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.864502 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6zhq\" (UniqueName: \"kubernetes.io/projected/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-kube-api-access-x6zhq\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965035 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.964993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965040 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965082 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-metrics-client-ca\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965172 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-grpc-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965224 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965209 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.965537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.965241 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6zhq\" (UniqueName: \"kubernetes.io/projected/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-kube-api-access-x6zhq\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.966300 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.966268 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-metrics-client-ca\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.968614 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.968565 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.968739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.968713 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.968791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.968749 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.969315 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.969290 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.969495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.969475 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.970034 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.970013 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-secret-grpc-tls\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:10.976342 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:10.976311 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6zhq\" (UniqueName: \"kubernetes.io/projected/0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e-kube-api-access-x6zhq\") pod \"thanos-querier-7d67f47b99-pbm6b\" (UID: \"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e\") " pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:11.054834 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:11.054725 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:11.076315 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:11.076272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-scwpv" event={"ID":"bc925f5a-6875-4f48-8be6-e2eca7639413","Type":"ContainerStarted","Data":"dbb9f9eeb6a18b538bf95ee30bc953d8ad14366133a9fc77e95aebc820ad29f7"} Jun 08 16:38:11.076757 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:11.076339 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-scwpv" event={"ID":"bc925f5a-6875-4f48-8be6-e2eca7639413","Type":"ContainerStarted","Data":"91b30d12db4996e326566519d325ad3c7fc1b4a4de067c15eea507b945b399f3"} Jun 08 16:38:11.100939 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:11.100885 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-scwpv" podStartSLOduration=2.891617685 podStartE2EDuration="4.10086811s" totalImagesPullingTime="1.132855586s" totalInitContainerRuntime="76.394839ms" isStatefulPod=true podCreationTimestamp="2026-06-08 16:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:11.100471259 +0000 UTC m=+295.486659653" watchObservedRunningTime="2026-06-08 16:38:11.10086811 +0000 UTC m=+295.487056504" Jun 08 16:38:11.206231 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:11.206169 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d67f47b99-pbm6b"] Jun 08 16:38:11.209348 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:11.209301 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1700cc_2287_4c4b_9c9b_aba2eaf6ee4e.slice/crio-e48164dd48cf25c488a6f7a5f56dc413a0bcbfd548852082a3fd60188fed75e1 WatchSource:0}: Error finding container e48164dd48cf25c488a6f7a5f56dc413a0bcbfd548852082a3fd60188fed75e1: Status 404 returned error can't find the container with id e48164dd48cf25c488a6f7a5f56dc413a0bcbfd548852082a3fd60188fed75e1 Jun 08 16:38:12.082074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.082033 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"e48164dd48cf25c488a6f7a5f56dc413a0bcbfd548852082a3fd60188fed75e1"} Jun 08 16:38:12.859183 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.859147 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-867f4f857-5hp5v"] Jun 08 16:38:12.863017 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.862977 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.866046 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866017 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 08 16:38:12.866197 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866132 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 08 16:38:12.866272 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866242 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 08 16:38:12.866543 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866525 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 08 16:38:12.866712 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866690 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 08 16:38:12.866869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.866851 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8l6hd\"" Jun 08 16:38:12.871132 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.871112 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 08 16:38:12.879740 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.879714 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-867f4f857-5hp5v"] Jun 08 16:38:12.985137 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985339 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp9bc\" (UniqueName: \"kubernetes.io/projected/64146db8-4329-413f-88d7-3f8d0bac5d8b-kube-api-access-fp9bc\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985339 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985339 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-serving-certs-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985506 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985364 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985506 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985390 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-federate-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985506 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985427 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:12.985506 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:12.985455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-metrics-client-ca\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.085914 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.085886 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp9bc\" (UniqueName: \"kubernetes.io/projected/64146db8-4329-413f-88d7-3f8d0bac5d8b-kube-api-access-fp9bc\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.085920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.085940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-serving-certs-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.085963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.085986 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-federate-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.086110 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.086151 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-metrics-client-ca\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086359 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.086215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086928 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.086814 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-serving-certs-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.086928 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.086814 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.087122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.087096 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64146db8-4329-413f-88d7-3f8d0bac5d8b-metrics-client-ca\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.088653 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.088616 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.088791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.088770 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.089191 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.089171 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-telemeter-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.089233 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.089182 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/64146db8-4329-413f-88d7-3f8d0bac5d8b-federate-client-tls\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.094350 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.094333 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp9bc\" (UniqueName: \"kubernetes.io/projected/64146db8-4329-413f-88d7-3f8d0bac5d8b-kube-api-access-fp9bc\") pod \"telemeter-client-867f4f857-5hp5v\" (UID: \"64146db8-4329-413f-88d7-3f8d0bac5d8b\") " pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.174923 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.174832 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" Jun 08 16:38:13.324160 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:13.324128 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-867f4f857-5hp5v"] Jun 08 16:38:13.328361 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:13.328335 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64146db8_4329_413f_88d7_3f8d0bac5d8b.slice/crio-82daebaf16ba40f354bb29d509b12e8fde9819ba91cb606dc9f7a185a602d591 WatchSource:0}: Error finding container 82daebaf16ba40f354bb29d509b12e8fde9819ba91cb606dc9f7a185a602d591: Status 404 returned error can't find the container with id 82daebaf16ba40f354bb29d509b12e8fde9819ba91cb606dc9f7a185a602d591 Jun 08 16:38:14.020599 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.020559 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:38:14.031667 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.031195 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.034534 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.034505 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 16:38:14.034703 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.034515 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 16:38:14.034703 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.034625 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 16:38:14.035047 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.035022 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 16:38:14.035198 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.035021 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 16:38:14.035625 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.035604 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 16:38:14.036029 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.035859 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 16:38:14.037446 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.036387 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 16:38:14.037446 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.036608 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.037584 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.038485 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.038744 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.039708 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-14nq9c0s5uoh3\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.039898 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-spk6p\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.040061 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 16:38:14.040783 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.040219 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 16:38:14.042650 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.042609 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 16:38:14.044303 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.044278 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:38:14.092773 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.092731 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" event={"ID":"64146db8-4329-413f-88d7-3f8d0bac5d8b","Type":"ContainerStarted","Data":"82daebaf16ba40f354bb29d509b12e8fde9819ba91cb606dc9f7a185a602d591"} Jun 08 16:38:14.096035 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.095841 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"f6ac3b5a23e00cba08345c617b3f02bfea81b25868abc0f9603974fb437cedaa"} Jun 08 16:38:14.096035 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.095930 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"f8c33e4e738658179d8078378a28a6bc9b32a01128015ec3257d39b70b77dce5"} Jun 08 16:38:14.096035 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.095975 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"281387227fd6fb3530d1385afb1281fed4dcc710cffbda4fdf7de8a2e2182a2a"} Jun 08 16:38:14.197369 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197375 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197426 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197452 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnj85\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197480 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197533 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.197570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197567 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197665 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197778 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197848 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197871 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197896 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198040 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.197938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.198033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.198085 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.198110 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.198463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.198147 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299137 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299182 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299218 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299279 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299301 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnj85\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299330 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299375 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299409 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299452 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299495 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299488 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299551 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299607 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299704 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299742 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299769 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.299894 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.299867 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.301127 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.300544 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.302144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.302112 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.302419 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.302339 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.302738 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.302712 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.302829 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.302781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.303742 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.303716 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.305065 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.305041 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.305194 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.305074 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.305428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.305402 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.306802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.306345 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.306802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.306493 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.306802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.306766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.306802 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.306774 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.308139 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.308096 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.309564 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.309371 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.310852 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.310811 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.310989 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.310965 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.311371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.311345 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.312144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.312122 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnj85\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85\") pod \"prometheus-k8s-0\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.348341 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.348249 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:14.526949 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:14.526914 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:38:14.530779 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:38:14.530714 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cea509_0ef8_4be4_aa0e_007a4d1527d6.slice/crio-e6d565bea47e2c1482e95398a98b47f45f5477c895760401cb733a688b520d6f WatchSource:0}: Error finding container e6d565bea47e2c1482e95398a98b47f45f5477c895760401cb733a688b520d6f: Status 404 returned error can't find the container with id e6d565bea47e2c1482e95398a98b47f45f5477c895760401cb733a688b520d6f Jun 08 16:38:15.045499 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.045463 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rbmgh" Jun 08 16:38:15.103683 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.103624 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"09782ff8ae3f63c41f58c877c2729eea8ea9b42bc8ade2a3d1b0e5519ba9d9b4"} Jun 08 16:38:15.104122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.103720 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"c99775ab8ee5bae9fa590f91dfa9e5dbe82c47448207fe5caff99bf686de0736"} Jun 08 16:38:15.104122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.103764 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" event={"ID":"0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e","Type":"ContainerStarted","Data":"b70af697abc1a6b86a73e2efec65871f9361aada7b1ff67e99c73f7171104a00"} Jun 08 16:38:15.104122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.103812 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:15.104873 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.104853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"e6d565bea47e2c1482e95398a98b47f45f5477c895760401cb733a688b520d6f"} Jun 08 16:38:15.125899 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:15.125852 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" podStartSLOduration=2.218430841 podStartE2EDuration="5.125836808s" totalImagesPullingTime="2.907405967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:38:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:15.124689386 +0000 UTC m=+299.510877806" watchObservedRunningTime="2026-06-08 16:38:15.125836808 +0000 UTC m=+299.512025202" Jun 08 16:38:16.060550 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.060465 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:38:16.060726 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.060715 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:38:16.066298 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.066280 2559 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 16:38:16.110786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.110607 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" event={"ID":"64146db8-4329-413f-88d7-3f8d0bac5d8b","Type":"ContainerStarted","Data":"7ae4dce23c6c0b7cba474ebca68dba4ef9577e4c2b2098e640b5f2e149fa2e43"} Jun 08 16:38:16.110786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.110739 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" event={"ID":"64146db8-4329-413f-88d7-3f8d0bac5d8b","Type":"ContainerStarted","Data":"7fa9e0d4c652d4748a0071776979482cab88f4128505ac1af1bbe84ec3f052d8"} Jun 08 16:38:16.110786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.110762 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" event={"ID":"64146db8-4329-413f-88d7-3f8d0bac5d8b","Type":"ContainerStarted","Data":"2e933fa4e43e36a1dbbad8abb7f12912e4b629be771e2ef74b98a75238c9976a"} Jun 08 16:38:16.131797 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.112116 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" exitCode=0 Jun 08 16:38:16.131797 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.117407 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} Jun 08 16:38:16.135467 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:16.135423 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-867f4f857-5hp5v" podStartSLOduration=1.881658244 podStartE2EDuration="4.135409338s" totalImagesPullingTime="2.253751094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:38:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:16.13503482 +0000 UTC m=+300.521223216" watchObservedRunningTime="2026-06-08 16:38:16.135409338 +0000 UTC m=+300.521597732" Jun 08 16:38:20.134860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:20.134830 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} Jun 08 16:38:20.135183 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:20.134893 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} Jun 08 16:38:20.135183 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:20.134920 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} Jun 08 16:38:20.135183 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:20.134954 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} Jun 08 16:38:20.135183 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:20.134976 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} Jun 08 16:38:21.118527 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:21.118498 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d67f47b99-pbm6b" Jun 08 16:38:21.142691 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:21.142612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerStarted","Data":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} Jun 08 16:38:21.196041 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:21.195985 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.431304188 podStartE2EDuration="8.195966781s" totalImagesPullingTime="4.742201521s" totalInitContainerRuntime="22.461072ms" isStatefulPod=false podCreationTimestamp="2026-06-08 16:38:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:38:21.193315441 +0000 UTC m=+305.579503860" watchObservedRunningTime="2026-06-08 16:38:21.195966781 +0000 UTC m=+305.582155175" Jun 08 16:38:21.319704 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:21.319664 2559 patch_prober.go:28] interesting pod/image-registry-555685f464-v6hcx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 16:38:21.319885 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:21.319719 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-555685f464-v6hcx" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 16:38:23.029866 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:23.029836 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:23.112068 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:23.112029 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:38:23.251689 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:23.251625 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:38:24.348471 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:24.348421 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:38:48.272171 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.272097 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-555685f464-v6hcx" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerName="registry" containerID="cri-o://0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2" gracePeriod=30 Jun 08 16:38:48.519684 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.519656 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:48.617460 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617416 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617485 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617539 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617558 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617581 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617698 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617690 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6hk6\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617971 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617726 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.617971 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.617774 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted\") pod \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\" (UID: \"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe\") " Jun 08 16:38:48.618157 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.618087 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:38:48.618401 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.618377 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:38:48.620876 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.620848 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:38:48.621001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.620869 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:38:48.621001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.620968 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:38:48.621001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.620984 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6" (OuterVolumeSpecName: "kube-api-access-k6hk6") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "kube-api-access-k6hk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:38:48.621115 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.621098 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:38:48.626973 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.626948 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" (UID: "d4a7ce8d-5e63-40db-bc8b-a94ce59753fe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 16:38:48.719358 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719320 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-trusted-ca\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719358 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719352 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-installation-pull-secrets\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719358 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719363 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-tls\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719596 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719371 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6hk6\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-kube-api-access-k6hk6\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719596 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719380 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-registry-certificates\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719596 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719388 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-ca-trust-extracted\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719596 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719397 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-image-registry-private-configuration\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:48.719596 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:48.719406 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe-bound-sa-token\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:38:49.248535 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.248498 2559 generic.go:356] "Generic (PLEG): container finished" podID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerID="0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2" exitCode=0 Jun 08 16:38:49.248763 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.248555 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-555685f464-v6hcx" Jun 08 16:38:49.248763 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.248585 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555685f464-v6hcx" event={"ID":"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe","Type":"ContainerDied","Data":"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2"} Jun 08 16:38:49.248763 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.248655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-555685f464-v6hcx" event={"ID":"d4a7ce8d-5e63-40db-bc8b-a94ce59753fe","Type":"ContainerDied","Data":"418715ea4a5f9e088676727d8f5446ac5014b3d85bcfd9e6a027c29b0571311d"} Jun 08 16:38:49.248763 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.248677 2559 scope.go:117] "RemoveContainer" containerID="0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2" Jun 08 16:38:49.257674 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.257646 2559 scope.go:117] "RemoveContainer" containerID="0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2" Jun 08 16:38:49.257956 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:38:49.257934 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2\": container with ID starting with 0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2 not found: ID does not exist" containerID="0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2" Jun 08 16:38:49.258039 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.257970 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2"} err="failed to get container status \"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2\": rpc error: code = NotFound desc = could not find container \"0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2\": container with ID starting with 0578ab5a6a041d1086898bb5f3102fc1adbdf6949e14602bbd0a404d542550a2 not found: ID does not exist" Jun 08 16:38:49.270724 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.270695 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:38:49.274549 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:49.274524 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-555685f464-v6hcx"] Jun 08 16:38:50.117177 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:50.117139 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" path="/var/lib/kubelet/pods/d4a7ce8d-5e63-40db-bc8b-a94ce59753fe/volumes" Jun 08 16:38:55.876319 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:55.876284 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmgh_a6083b35-bb54-4772-b8ff-93f4bdbab53c/dns/0.log" Jun 08 16:38:56.072836 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:56.072809 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmgh_a6083b35-bb54-4772-b8ff-93f4bdbab53c/kube-rbac-proxy/0.log" Jun 08 16:38:56.473188 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:56.473160 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kxw8p_1dedd458-09c2-40f1-9dfa-2bd02e1f8965/dns-node-resolver/0.log" Jun 08 16:38:56.875978 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:56.875953 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 16:38:57.072875 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:38:57.072847 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-72zpv_3b1c4918-0486-4c55-9efa-83db938280a8/serve-healthcheck-canary/0.log" Jun 08 16:39:14.349441 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:14.349347 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:14.365658 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:14.365606 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:15.364732 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:15.364705 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:30.080420 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.080358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:39:30.082976 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.082947 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f972084f-acf6-4427-b7ea-0f340e15343b-metrics-certs\") pod \"network-metrics-daemon-5z57p\" (UID: \"f972084f-acf6-4427-b7ea-0f340e15343b\") " pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:39:30.315759 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.315726 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mvqv5\"" Jun 08 16:39:30.323118 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.323090 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5z57p" Jun 08 16:39:30.461841 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.461817 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5z57p"] Jun 08 16:39:30.464438 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:39:30.464409 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf972084f_acf6_4427_b7ea_0f340e15343b.slice/crio-fec5499f8da0d31af1dc00288aee17c763d01744ab85d2324528c082d97695ee WatchSource:0}: Error finding container fec5499f8da0d31af1dc00288aee17c763d01744ab85d2324528c082d97695ee: Status 404 returned error can't find the container with id fec5499f8da0d31af1dc00288aee17c763d01744ab85d2324528c082d97695ee Jun 08 16:39:30.466441 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:30.466422 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:39:31.411337 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:31.411295 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5z57p" event={"ID":"f972084f-acf6-4427-b7ea-0f340e15343b","Type":"ContainerStarted","Data":"fec5499f8da0d31af1dc00288aee17c763d01744ab85d2324528c082d97695ee"} Jun 08 16:39:32.418539 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.418499 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5z57p" event={"ID":"f972084f-acf6-4427-b7ea-0f340e15343b","Type":"ContainerStarted","Data":"890c6def01715b736bbef9b1a7f0fa57b9fca262a1b0427daef205af723c4722"} Jun 08 16:39:32.418941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.418565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5z57p" event={"ID":"f972084f-acf6-4427-b7ea-0f340e15343b","Type":"ContainerStarted","Data":"1e8942cd9f2c224a54b1d0c3e03d47e5ccada642da97174aac6ec5d55e5a3d29"} Jun 08 16:39:32.427796 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.427763 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:32.428113 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428077 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="prometheus" containerID="cri-o://225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" gracePeriod=600 Jun 08 16:39:32.428284 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428094 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="thanos-sidecar" containerID="cri-o://810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" gracePeriod=600 Jun 08 16:39:32.428353 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428098 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="config-reloader" containerID="cri-o://4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" gracePeriod=600 Jun 08 16:39:32.428461 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428094 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy" containerID="cri-o://639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" gracePeriod=600 Jun 08 16:39:32.428525 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428119 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-web" containerID="cri-o://e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" gracePeriod=600 Jun 08 16:39:32.428583 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.428133 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" gracePeriod=600 Jun 08 16:39:32.440510 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.440456 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5z57p" podStartSLOduration=375.519441614 podStartE2EDuration="6m16.440437198s" totalImagesPullingTime="920.995584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:39:32.439129577 +0000 UTC m=+376.825317997" watchObservedRunningTime="2026-06-08 16:39:32.440437198 +0000 UTC m=+376.826625589" Jun 08 16:39:32.691612 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.691584 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:32.803463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803423 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803460 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803477 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803512 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803537 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803562 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803582 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803598 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803623 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803677 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803724 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.803764 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803751 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803779 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803804 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnj85\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803830 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803872 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803866 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803906 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803934 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803944 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.803996 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804009 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804234 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804048 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs\") pod \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\" (UID: \"f6cea509-0ef8-4be4-aa0e-007a4d1527d6\") " Jun 08 16:39:32.804778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804283 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.804778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804356 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-metrics-client-ca\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.804778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804374 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.804778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804387 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.804778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.804403 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.805202 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.805124 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.806512 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.805447 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.806512 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.805839 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:39:32.806841 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.806769 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 16:39:32.807262 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.807146 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out" (OuterVolumeSpecName: "config-out") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 16:39:32.807262 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.807232 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.807529 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.807499 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.808665 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.808616 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.808779 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.808754 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.808846 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.808771 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.808846 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.808787 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.808968 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.808951 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:39:32.809159 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.809141 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.809436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.809418 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85" (OuterVolumeSpecName: "kube-api-access-cnj85") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "kube-api-access-cnj85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:39:32.809506 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.809483 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config" (OuterVolumeSpecName: "config") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.819675 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.819624 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config" (OuterVolumeSpecName: "web-config") pod "f6cea509-0ef8-4be4-aa0e-007a4d1527d6" (UID: "f6cea509-0ef8-4be4-aa0e-007a4d1527d6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:39:32.904975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.904931 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.904975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.904968 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.904975 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.904982 2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-web-config\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.904997 2559 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-metrics-client-certs\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905010 2559 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-kube-rbac-proxy\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905022 2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config-out\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905033 2559 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905043 2559 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-grpc-tls\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905053 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905065 2559 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905078 2559 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905091 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905103 2559 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-prometheus-k8s-db\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905116 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnj85\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-kube-api-access-cnj85\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905127 2559 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-config\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:32.905245 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:32.905139 2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f6cea509-0ef8-4be4-aa0e-007a4d1527d6-tls-assets\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:39:33.425318 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425279 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" exitCode=0 Jun 08 16:39:33.425318 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425310 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" exitCode=0 Jun 08 16:39:33.425318 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425318 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" exitCode=0 Jun 08 16:39:33.425318 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425327 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" exitCode=0 Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425335 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" exitCode=0 Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425345 2559 generic.go:356] "Generic (PLEG): container finished" podID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" exitCode=0 Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425356 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425381 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425439 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425469 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425503 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425506 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425544 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} Jun 08 16:39:33.425858 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.425624 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f6cea509-0ef8-4be4-aa0e-007a4d1527d6","Type":"ContainerDied","Data":"e6d565bea47e2c1482e95398a98b47f45f5477c895760401cb733a688b520d6f"} Jun 08 16:39:33.434170 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.434056 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.441677 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.441659 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.450543 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.450500 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:33.456438 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.456418 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:33.460144 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.460121 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.467134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.467115 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.474408 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.474391 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.480372 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480351 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:33.480728 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480711 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="config-reloader" Jun 08 16:39:33.480780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480734 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="config-reloader" Jun 08 16:39:33.480780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480749 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="thanos-sidecar" Jun 08 16:39:33.480780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480754 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="thanos-sidecar" Jun 08 16:39:33.480780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480774 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="init-config-reloader" Jun 08 16:39:33.480780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480779 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="init-config-reloader" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480787 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="prometheus" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480793 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="prometheus" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480799 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480804 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480812 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerName="registry" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480818 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerName="registry" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480826 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-web" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480831 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-web" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480837 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-thanos" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480842 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-thanos" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480892 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="thanos-sidecar" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480902 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-web" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480908 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy-thanos" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480915 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="kube-rbac-proxy" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480920 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a7ce8d-5e63-40db-bc8b-a94ce59753fe" containerName="registry" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480925 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="config-reloader" Jun 08 16:39:33.481006 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.480931 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" containerName="prometheus" Jun 08 16:39:33.483415 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.483391 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.486065 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.486048 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.488748 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.488725 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 08 16:39:33.488837 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.488725 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 08 16:39:33.488959 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.488943 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-spk6p\"" Jun 08 16:39:33.489120 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489099 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 08 16:39:33.489306 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489124 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 08 16:39:33.489306 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489226 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 08 16:39:33.489306 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489225 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 08 16:39:33.489739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489717 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 08 16:39:33.491003 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489806 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 08 16:39:33.491003 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489958 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 08 16:39:33.491003 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489864 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 08 16:39:33.491003 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.489872 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 08 16:39:33.492866 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.491877 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 08 16:39:33.492866 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.492119 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-14nq9c0s5uoh3\"" Jun 08 16:39:33.492866 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.492289 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 08 16:39:33.493401 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.493185 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.493650 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.493604 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.493748 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.493656 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.493748 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.493680 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.493935 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.493918 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 08 16:39:33.494016 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.493991 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.494054 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494023 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.494054 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494040 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.494305 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.494284 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.494406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494310 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.494406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494337 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.494622 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.494600 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.494702 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494663 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.494702 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494685 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.494940 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.494918 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.495011 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494943 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.495011 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.494959 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.495254 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.495220 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.495377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495260 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.495377 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495280 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.495588 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:39:33.495569 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.495690 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495592 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.495690 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495605 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.495960 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495925 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.496098 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.495963 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.496283 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.496259 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.496336 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.496286 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.496851 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.496666 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 08 16:39:33.496851 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.496657 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.496851 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.496700 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.497066 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497044 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.497133 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497068 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.497307 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497287 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.497373 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497306 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.497549 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497530 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.497597 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497550 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.497804 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497785 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.497873 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497806 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.498018 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.497999 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.498092 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498017 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.498239 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498222 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.498295 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498238 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.498442 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498424 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.498507 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498443 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.498741 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498721 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.498795 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498742 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.498944 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498923 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.499008 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.498946 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.499162 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499145 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.499223 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499162 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.499366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499350 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.499412 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499365 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.499569 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499551 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.499730 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499571 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.499886 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499870 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.499934 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.499886 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.500087 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500070 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.500161 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500090 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.500303 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500288 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.500346 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500302 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.500472 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500457 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.500518 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500471 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.500684 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500665 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.500731 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500684 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.500889 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500872 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.500930 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.500890 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.501070 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501053 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.501140 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501072 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.501277 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501261 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.501320 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501276 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.501492 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501475 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.501537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501492 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.501688 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501671 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.501739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501689 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.501880 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501857 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.501923 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.501883 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.502087 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502072 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.502130 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502086 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.502302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502287 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.502302 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502301 2559 scope.go:117] "RemoveContainer" containerID="154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3" Jun 08 16:39:33.502491 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502475 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3"} err="failed to get container status \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": rpc error: code = NotFound desc = could not find container \"154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3\": container with ID starting with 154564c859e864da240b7cf812830672f9d51a1868c0b3770ea4f4efac20dbf3 not found: ID does not exist" Jun 08 16:39:33.502529 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502492 2559 scope.go:117] "RemoveContainer" containerID="639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284" Jun 08 16:39:33.502735 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502717 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284"} err="failed to get container status \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": rpc error: code = NotFound desc = could not find container \"639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284\": container with ID starting with 639a3d5fbd470dfa5ea7af5d895dc20c898076c886844212be4a6cc591939284 not found: ID does not exist" Jun 08 16:39:33.502791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502735 2559 scope.go:117] "RemoveContainer" containerID="e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3" Jun 08 16:39:33.503015 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.502994 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3"} err="failed to get container status \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": rpc error: code = NotFound desc = could not find container \"e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3\": container with ID starting with e18bd662bce95f88f9c071a03585d637beb04e0eb00142d240d4a21b1bef5ed3 not found: ID does not exist" Jun 08 16:39:33.503015 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503014 2559 scope.go:117] "RemoveContainer" containerID="810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a" Jun 08 16:39:33.503306 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503286 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a"} err="failed to get container status \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": rpc error: code = NotFound desc = could not find container \"810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a\": container with ID starting with 810f68834290b065e8da870a905f3327a053b4b4e09f604ba35e2830e490813a not found: ID does not exist" Jun 08 16:39:33.503406 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503308 2559 scope.go:117] "RemoveContainer" containerID="4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe" Jun 08 16:39:33.503702 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503669 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe"} err="failed to get container status \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": rpc error: code = NotFound desc = could not find container \"4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe\": container with ID starting with 4120eb7697b7bd4b604e9cfe08b813686385357151851fb8a5b473dc56177bbe not found: ID does not exist" Jun 08 16:39:33.503702 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503702 2559 scope.go:117] "RemoveContainer" containerID="225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a" Jun 08 16:39:33.504020 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.503999 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a"} err="failed to get container status \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": rpc error: code = NotFound desc = could not find container \"225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a\": container with ID starting with 225ad12a2ce54a26cb7b2b94c68b9506fb350d16e82d33c5604e62324aa8361a not found: ID does not exist" Jun 08 16:39:33.504140 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.504021 2559 scope.go:117] "RemoveContainer" containerID="716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace" Jun 08 16:39:33.504285 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.504267 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace"} err="failed to get container status \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": rpc error: code = NotFound desc = could not find container \"716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace\": container with ID starting with 716376f8ac5927f9ba053ba1c1b37e53569fb9be7a59db0778c20b7fd3ce4ace not found: ID does not exist" Jun 08 16:39:33.505548 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.505529 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:33.611309 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611264 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611309 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611359 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611400 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-config-out\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611418 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611438 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611457 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611526 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611475 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611534 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-web-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611577 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611607 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611645 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611668 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611695 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9s98\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-kube-api-access-x9s98\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611717 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611926 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611735 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611926 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611757 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611926 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611786 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611926 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611804 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.611926 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.611839 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712583 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712490 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712583 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712583 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712546 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712583 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712570 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-web-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712592 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9s98\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-kube-api-access-x9s98\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712768 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712799 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712828 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712862 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712899 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712925 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.712936 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.713705 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.713705 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.712984 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-config-out\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.713705 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.713004 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.713894 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.714092 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.714249 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.714440 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.714909 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716032 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.715723 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.716193 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-web-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.716221 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716421 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.716348 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.716624 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.716569 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.719791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.719201 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.719791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.719258 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.719791 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.719766 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.720013 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.719907 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e52f2e78-82c6-410b-b523-4cbe18750ed5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.720013 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.719761 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.720107 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.720081 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.720157 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.720118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e52f2e78-82c6-410b-b523-4cbe18750ed5-config-out\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.720371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.720351 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e52f2e78-82c6-410b-b523-4cbe18750ed5-config\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.722190 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.722162 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.726025 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.725999 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9s98\" (UniqueName: \"kubernetes.io/projected/e52f2e78-82c6-410b-b523-4cbe18750ed5-kube-api-access-x9s98\") pod \"prometheus-k8s-0\" (UID: \"e52f2e78-82c6-410b-b523-4cbe18750ed5\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.800104 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.800059 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:39:33.946393 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:33.946360 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 08 16:39:33.949169 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:39:33.949140 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52f2e78_82c6_410b_b523_4cbe18750ed5.slice/crio-11884f90b746700b1d1d3393c8b73333cccab8d75dc789e4d4e14d90fcce1763 WatchSource:0}: Error finding container 11884f90b746700b1d1d3393c8b73333cccab8d75dc789e4d4e14d90fcce1763: Status 404 returned error can't find the container with id 11884f90b746700b1d1d3393c8b73333cccab8d75dc789e4d4e14d90fcce1763 Jun 08 16:39:34.118054 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:34.118021 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6cea509-0ef8-4be4-aa0e-007a4d1527d6" path="/var/lib/kubelet/pods/f6cea509-0ef8-4be4-aa0e-007a4d1527d6/volumes" Jun 08 16:39:34.431867 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:34.431831 2559 generic.go:356] "Generic (PLEG): container finished" podID="e52f2e78-82c6-410b-b523-4cbe18750ed5" containerID="cd0ab2099f7e10f97e809f5359395306bfb7945603b5f9baa938ced09e98e194" exitCode=0 Jun 08 16:39:34.432292 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:34.431925 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerDied","Data":"cd0ab2099f7e10f97e809f5359395306bfb7945603b5f9baa938ced09e98e194"} Jun 08 16:39:34.432292 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:34.432003 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"11884f90b746700b1d1d3393c8b73333cccab8d75dc789e4d4e14d90fcce1763"} Jun 08 16:39:35.440351 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440313 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"9ff3c9f781ea5599f5f7e906876fd00aaec5e3a64567602f79bbc8a7291640fd"} Jun 08 16:39:35.440780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440397 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"ab67cb1870bdc87a6d7de8759c88064b9e86e19631ce47cdf89a2699a43a10e6"} Jun 08 16:39:35.440780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440440 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"ce4354cbe7abc440b2b29a602af6d9318bb65c37f8c8cdf453d934defdca16a8"} Jun 08 16:39:35.440780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440476 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"a44ef4074d771456b4ba150e08e58d9b0607b47290e29757929fdc7c1affeda9"} Jun 08 16:39:35.440780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440515 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"c3053030229cb569d0b60488415fd86cf24ed172349b955ae2ba348b1b0f9c97"} Jun 08 16:39:35.440780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.440551 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e52f2e78-82c6-410b-b523-4cbe18750ed5","Type":"ContainerStarted","Data":"4f770671812f38724a878c5e63fa8cc478f09f3c1fa0886854c90daccfc3c050"} Jun 08 16:39:35.469968 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:35.469898 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.450818026 podStartE2EDuration="2.469875311s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.057285ms" isStatefulPod=false podCreationTimestamp="2026-06-08 16:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:39:35.467991784 +0000 UTC m=+379.854180177" watchObservedRunningTime="2026-06-08 16:39:35.469875311 +0000 UTC m=+379.856063707" Jun 08 16:39:38.800824 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:39:38.800782 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:40:33.800570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:40:33.800474 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:40:33.816983 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:40:33.816957 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:40:34.691392 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:40:34.691362 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 08 16:43:16.085381 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:16.085340 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:43:16.086066 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:16.086045 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:43:28.622141 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.622104 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8"] Jun 08 16:43:28.625540 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.625521 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.630103 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.630077 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 16:43:28.630263 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.630111 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 16:43:28.630708 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.630694 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 16:43:28.630922 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.630908 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mr7dg\"" Jun 08 16:43:28.631018 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.631002 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 16:43:28.661097 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.661063 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8"] Jun 08 16:43:28.791286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.791248 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-manifests\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.791286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.791283 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.791537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.791309 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-tmp\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.791537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.791429 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4js\" (UniqueName: \"kubernetes.io/projected/dfbf5980-9001-430b-abe8-c57cb88c4dc5-kube-api-access-wq4js\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.791537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.791509 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.892780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.892676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4js\" (UniqueName: \"kubernetes.io/projected/dfbf5980-9001-430b-abe8-c57cb88c4dc5-kube-api-access-wq4js\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.892780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.892735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.892780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.892772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-manifests\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.893069 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.892789 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.893069 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.892811 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-tmp\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.893225 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.893200 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-manifests\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.895364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.895328 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dfbf5980-9001-430b-abe8-c57cb88c4dc5-tmp\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.895483 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.895335 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.895527 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.895511 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dfbf5980-9001-430b-abe8-c57cb88c4dc5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.901840 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.901814 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4js\" (UniqueName: \"kubernetes.io/projected/dfbf5980-9001-430b-abe8-c57cb88c4dc5-kube-api-access-wq4js\") pod \"opendatahub-operator-controller-manager-bf5fd9654-ht8h8\" (UID: \"dfbf5980-9001-430b-abe8-c57cb88c4dc5\") " pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:28.935830 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:28.935790 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:29.079883 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:29.079843 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8"] Jun 08 16:43:29.084581 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:43:29.084552 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfbf5980_9001_430b_abe8_c57cb88c4dc5.slice/crio-422fc580b9938e4d75d2d90be2e51ebc4ccf6123f4ccbf098be3dc9ffd85993d WatchSource:0}: Error finding container 422fc580b9938e4d75d2d90be2e51ebc4ccf6123f4ccbf098be3dc9ffd85993d: Status 404 returned error can't find the container with id 422fc580b9938e4d75d2d90be2e51ebc4ccf6123f4ccbf098be3dc9ffd85993d Jun 08 16:43:29.355541 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:29.355507 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" event={"ID":"dfbf5980-9001-430b-abe8-c57cb88c4dc5","Type":"ContainerStarted","Data":"422fc580b9938e4d75d2d90be2e51ebc4ccf6123f4ccbf098be3dc9ffd85993d"} Jun 08 16:43:32.371748 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:32.371705 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" event={"ID":"dfbf5980-9001-430b-abe8-c57cb88c4dc5","Type":"ContainerStarted","Data":"c1b046ecc314a8b20686ba9bdc698cff6de0e2af0527474bced498e5a96121f9"} Jun 08 16:43:33.377276 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:33.377238 2559 generic.go:356] "Generic (PLEG): container finished" podID="dfbf5980-9001-430b-abe8-c57cb88c4dc5" containerID="c1b046ecc314a8b20686ba9bdc698cff6de0e2af0527474bced498e5a96121f9" exitCode=0 Jun 08 16:43:33.377718 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:33.377334 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" event={"ID":"dfbf5980-9001-430b-abe8-c57cb88c4dc5","Type":"ContainerDied","Data":"c1b046ecc314a8b20686ba9bdc698cff6de0e2af0527474bced498e5a96121f9"} Jun 08 16:43:34.382905 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:34.382873 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" event={"ID":"dfbf5980-9001-430b-abe8-c57cb88c4dc5","Type":"ContainerStarted","Data":"545e88403872420cb380a4e6220cfca81e299f7111dea1925817ea604c395ac4"} Jun 08 16:43:34.383261 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:34.383009 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:34.407030 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:34.406971 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" podStartSLOduration=2.246142978 podStartE2EDuration="6.406948538s" totalImagesPullingTime="3.092421058s" totalInitContainerRuntime="1.068384502s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:43:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:43:34.405545765 +0000 UTC m=+618.791734177" watchObservedRunningTime="2026-06-08 16:43:34.406948538 +0000 UTC m=+618.793136932" Jun 08 16:43:45.389532 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:45.389497 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bf5fd9654-ht8h8" Jun 08 16:43:50.117699 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.117669 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-2hkjc"] Jun 08 16:43:50.129793 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.129763 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.132703 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.132678 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 08 16:43:50.133134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.133114 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2wrkx\"" Jun 08 16:43:50.139242 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.139214 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-2hkjc"] Jun 08 16:43:50.165149 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.165116 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqp8v\" (UniqueName: \"kubernetes.io/projected/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-kube-api-access-fqp8v\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.165342 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.165194 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-tls-certs\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.265839 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.265793 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqp8v\" (UniqueName: \"kubernetes.io/projected/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-kube-api-access-fqp8v\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.266005 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.265851 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-tls-certs\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.268563 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.268523 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-tls-certs\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.275014 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.274982 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqp8v\" (UniqueName: \"kubernetes.io/projected/2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139-kube-api-access-fqp8v\") pod \"model-serving-api-5dd488fc48-2hkjc\" (UID: \"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139\") " pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.442271 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.442174 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:50.584697 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:50.584664 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-2hkjc"] Jun 08 16:43:50.588014 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:43:50.587985 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfc5d20_4fad_42ec_9fc7_3ae3c5c4d139.slice/crio-4a53f04442b192e56b8449aa3509503b2d962fa561a3da28e41f0c6c5bede6c5 WatchSource:0}: Error finding container 4a53f04442b192e56b8449aa3509503b2d962fa561a3da28e41f0c6c5bede6c5: Status 404 returned error can't find the container with id 4a53f04442b192e56b8449aa3509503b2d962fa561a3da28e41f0c6c5bede6c5 Jun 08 16:43:51.453088 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:51.453043 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" event={"ID":"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139","Type":"ContainerStarted","Data":"4a53f04442b192e56b8449aa3509503b2d962fa561a3da28e41f0c6c5bede6c5"} Jun 08 16:43:53.464305 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:53.464272 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" event={"ID":"2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139","Type":"ContainerStarted","Data":"fa4c5db3acc2f380a0b1a4138c44c29cd381e3b22c4ea221d538bf4d41662d30"} Jun 08 16:43:53.464741 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:53.464338 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:43:53.484722 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:53.484674 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" podStartSLOduration=1.108463793 podStartE2EDuration="3.484626395s" totalImagesPullingTime="2.376162602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:43:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:43:53.482565702 +0000 UTC m=+637.868754095" watchObservedRunningTime="2026-06-08 16:43:53.484626395 +0000 UTC m=+637.870814808" Jun 08 16:43:57.440961 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.440922 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g"] Jun 08 16:43:57.444418 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.444399 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.447444 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.447420 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 08 16:43:57.448974 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.448954 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 08 16:43:57.449082 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.448958 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 08 16:43:57.449251 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.449231 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 08 16:43:57.449386 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.449241 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 08 16:43:57.449386 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.449323 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-scxgn\"" Jun 08 16:43:57.455078 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.455054 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g"] Jun 08 16:43:57.528739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.528708 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-metrics-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.528915 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.528747 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wthh5\" (UniqueName: \"kubernetes.io/projected/c8161c8e-c54c-49ac-8796-f31c339c8746-kube-api-access-wthh5\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.528915 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.528888 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.529034 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.528944 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c8161c8e-c54c-49ac-8796-f31c339c8746-manager-config\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.630058 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.630012 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.630230 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.630073 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c8161c8e-c54c-49ac-8796-f31c339c8746-manager-config\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.630230 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.630185 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-metrics-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.630230 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.630225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wthh5\" (UniqueName: \"kubernetes.io/projected/c8161c8e-c54c-49ac-8796-f31c339c8746-kube-api-access-wthh5\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.630823 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.630798 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c8161c8e-c54c-49ac-8796-f31c339c8746-manager-config\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.632798 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.632779 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-metrics-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.632869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.632782 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8161c8e-c54c-49ac-8796-f31c339c8746-cert\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.655275 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.655247 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wthh5\" (UniqueName: \"kubernetes.io/projected/c8161c8e-c54c-49ac-8796-f31c339c8746-kube-api-access-wthh5\") pod \"lws-controller-manager-5b766c7786-hqp8g\" (UID: \"c8161c8e-c54c-49ac-8796-f31c339c8746\") " pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.755740 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.755618 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:43:57.891566 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:57.891537 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g"] Jun 08 16:43:57.894138 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:43:57.894111 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8161c8e_c54c_49ac_8796_f31c339c8746.slice/crio-5250f35cc555454391336f60df19cb1557391caf46f21ca73ab6156502d419be WatchSource:0}: Error finding container 5250f35cc555454391336f60df19cb1557391caf46f21ca73ab6156502d419be: Status 404 returned error can't find the container with id 5250f35cc555454391336f60df19cb1557391caf46f21ca73ab6156502d419be Jun 08 16:43:58.488286 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:43:58.488249 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" event={"ID":"c8161c8e-c54c-49ac-8796-f31c339c8746","Type":"ContainerStarted","Data":"5250f35cc555454391336f60df19cb1557391caf46f21ca73ab6156502d419be"} Jun 08 16:44:01.503906 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:01.503864 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" event={"ID":"c8161c8e-c54c-49ac-8796-f31c339c8746","Type":"ContainerStarted","Data":"7156115a90975006b216faef720f78bce08eeb3494f8b5b5d089b5e16786aa05"} Jun 08 16:44:01.504353 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:01.503939 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:44:01.572227 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:01.572156 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" podStartSLOduration=1.853735789 podStartE2EDuration="4.572133445s" totalImagesPullingTime="2.718397656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:43:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:44:01.569205447 +0000 UTC m=+645.955393845" watchObservedRunningTime="2026-06-08 16:44:01.572133445 +0000 UTC m=+645.958321840" Jun 08 16:44:04.473445 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:04.473415 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-2hkjc" Jun 08 16:44:12.511571 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:12.511534 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b766c7786-hqp8g" Jun 08 16:44:20.435299 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.435251 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-75b7f6c686-8s9lc"] Jun 08 16:44:20.440336 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.440307 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.444162 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.444132 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hqstp\"" Jun 08 16:44:20.444574 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.444555 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 08 16:44:20.449778 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.449751 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-75b7f6c686-8s9lc"] Jun 08 16:44:20.543355 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.543302 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54f66\" (UniqueName: \"kubernetes.io/projected/5261595f-b064-4015-bada-3748f3fb6fd5-kube-api-access-54f66\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.543564 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.543397 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5261595f-b064-4015-bada-3748f3fb6fd5-redirect-config\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.643982 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.643922 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5261595f-b064-4015-bada-3748f3fb6fd5-redirect-config\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.644170 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.644003 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54f66\" (UniqueName: \"kubernetes.io/projected/5261595f-b064-4015-bada-3748f3fb6fd5-kube-api-access-54f66\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.644660 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.644608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5261595f-b064-4015-bada-3748f3fb6fd5-redirect-config\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.653025 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.652994 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54f66\" (UniqueName: \"kubernetes.io/projected/5261595f-b064-4015-bada-3748f3fb6fd5-kube-api-access-54f66\") pod \"dashboard-redirect-75b7f6c686-8s9lc\" (UID: \"5261595f-b064-4015-bada-3748f3fb6fd5\") " pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.752695 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.752603 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" Jun 08 16:44:20.899540 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:20.899516 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-75b7f6c686-8s9lc"] Jun 08 16:44:20.901537 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:44:20.901513 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5261595f_b064_4015_bada_3748f3fb6fd5.slice/crio-53849004601dd17a89ce44df7994d5b77de274d092d2f1572d33dd99ca3744b3 WatchSource:0}: Error finding container 53849004601dd17a89ce44df7994d5b77de274d092d2f1572d33dd99ca3744b3: Status 404 returned error can't find the container with id 53849004601dd17a89ce44df7994d5b77de274d092d2f1572d33dd99ca3744b3 Jun 08 16:44:21.592790 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:21.592746 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" event={"ID":"5261595f-b064-4015-bada-3748f3fb6fd5","Type":"ContainerStarted","Data":"53849004601dd17a89ce44df7994d5b77de274d092d2f1572d33dd99ca3744b3"} Jun 08 16:44:23.603352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:23.603313 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" event={"ID":"5261595f-b064-4015-bada-3748f3fb6fd5","Type":"ContainerStarted","Data":"5c66cc2cb0b401596c4b02ec92c7e9d2c969a6c024b9ea8cb11fe9913333f95b"} Jun 08 16:44:23.625645 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:23.625585 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-75b7f6c686-8s9lc" podStartSLOduration=1.886004223 podStartE2EDuration="3.62557135s" totalImagesPullingTime="1.739567127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:44:23.624014307 +0000 UTC m=+668.010202700" watchObservedRunningTime="2026-06-08 16:44:23.62557135 +0000 UTC m=+668.011759743" Jun 08 16:44:25.098627 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.098592 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b"] Jun 08 16:44:25.101490 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.101471 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.105423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.105397 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 08 16:44:25.106767 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.106744 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pwtvg\"" Jun 08 16:44:25.130851 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.130816 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b"] Jun 08 16:44:25.186304 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186304 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186315 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186423 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186452 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186481 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186500 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx7xm\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-kube-api-access-vx7xm\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186563 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.186860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.186658 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.287869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.287829 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.287882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.287903 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.287935 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.287965 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288005 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288028 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288074 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288083 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx7xm\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-kube-api-access-vx7xm\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288483 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288701 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288671 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288829 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288767 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288829 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288779 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.288829 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.288805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.290595 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.290573 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.290839 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.290822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.304476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.304443 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.304589 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.304523 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx7xm\" (UniqueName: \"kubernetes.io/projected/90bf27fa-47f1-4f79-9e73-44f54baf2ec4-kube-api-access-vx7xm\") pod \"data-science-gateway-data-science-gateway-class-74759f9795jnn2b\" (UID: \"90bf27fa-47f1-4f79-9e73-44f54baf2ec4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.412533 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.412435 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:25.609342 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:25.609288 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b"] Jun 08 16:44:25.611996 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:44:25.611968 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90bf27fa_47f1_4f79_9e73_44f54baf2ec4.slice/crio-4da225706fee3064ab80dbadef31d3a18aae8e3f615844db3a2d6f8bcfccacda WatchSource:0}: Error finding container 4da225706fee3064ab80dbadef31d3a18aae8e3f615844db3a2d6f8bcfccacda: Status 404 returned error can't find the container with id 4da225706fee3064ab80dbadef31d3a18aae8e3f615844db3a2d6f8bcfccacda Jun 08 16:44:26.619294 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:26.619251 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" event={"ID":"90bf27fa-47f1-4f79-9e73-44f54baf2ec4","Type":"ContainerStarted","Data":"4da225706fee3064ab80dbadef31d3a18aae8e3f615844db3a2d6f8bcfccacda"} Jun 08 16:44:28.296914 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:28.296871 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:44:28.297281 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:28.296963 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:44:28.297281 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:28.297010 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:44:28.632386 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:28.632353 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" event={"ID":"90bf27fa-47f1-4f79-9e73-44f54baf2ec4","Type":"ContainerStarted","Data":"c93007bb8e1b26f242060eaad921fe6f8f98ed59a55e912d5b5580f0d79baa84"} Jun 08 16:44:28.655911 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:28.655856 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" podStartSLOduration=0.973053299 podStartE2EDuration="3.65583946s" totalImagesPullingTime="2.682786161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:44:28.654438407 +0000 UTC m=+673.040626823" watchObservedRunningTime="2026-06-08 16:44:28.65583946 +0000 UTC m=+673.042027853" Jun 08 16:44:29.413226 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:29.413186 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:29.418266 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:29.418237 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:29.637380 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:29.637337 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:29.638272 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:29.638255 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795jnn2b" Jun 08 16:44:50.184202 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.184161 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:50.187972 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.187943 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:50.191460 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.191428 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 16:44:50.191619 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.191475 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-q85bm\"" Jun 08 16:44:50.192436 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.192416 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 16:44:50.197600 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.197566 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:50.323825 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.323781 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rr94\" (UniqueName: \"kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94\") pod \"kuadrant-operator-catalog-j9x7v\" (UID: \"ca7d63df-0262-4500-b5cd-d42382363201\") " pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:50.425170 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.425133 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rr94\" (UniqueName: \"kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94\") pod \"kuadrant-operator-catalog-j9x7v\" (UID: \"ca7d63df-0262-4500-b5cd-d42382363201\") " pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:50.434423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.434356 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rr94\" (UniqueName: \"kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94\") pod \"kuadrant-operator-catalog-j9x7v\" (UID: \"ca7d63df-0262-4500-b5cd-d42382363201\") " pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:50.501729 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.501698 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:50.558353 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.558320 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:50.639773 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.639588 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:50.642670 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:44:50.642626 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca7d63df_0262_4500_b5cd_d42382363201.slice/crio-82d4e50019dfd9ae6961256759eeee2e8074f37371bd47ac74d8b9051e5d6d66 WatchSource:0}: Error finding container 82d4e50019dfd9ae6961256759eeee2e8074f37371bd47ac74d8b9051e5d6d66: Status 404 returned error can't find the container with id 82d4e50019dfd9ae6961256759eeee2e8074f37371bd47ac74d8b9051e5d6d66 Jun 08 16:44:50.643979 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.643958 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:44:50.732395 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.732300 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" event={"ID":"ca7d63df-0262-4500-b5cd-d42382363201","Type":"ContainerStarted","Data":"82d4e50019dfd9ae6961256759eeee2e8074f37371bd47ac74d8b9051e5d6d66"} Jun 08 16:44:50.760191 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.760149 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m56xq"] Jun 08 16:44:50.764621 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.764596 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:44:50.772652 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.772608 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m56xq"] Jun 08 16:44:50.828738 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.828692 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kvsj\" (UniqueName: \"kubernetes.io/projected/7766c1c7-0747-4391-af89-7ea28f9d9fb6-kube-api-access-8kvsj\") pod \"kuadrant-operator-catalog-m56xq\" (UID: \"7766c1c7-0747-4391-af89-7ea28f9d9fb6\") " pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:44:50.929924 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.929883 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kvsj\" (UniqueName: \"kubernetes.io/projected/7766c1c7-0747-4391-af89-7ea28f9d9fb6-kube-api-access-8kvsj\") pod \"kuadrant-operator-catalog-m56xq\" (UID: \"7766c1c7-0747-4391-af89-7ea28f9d9fb6\") " pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:44:50.939357 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:50.939326 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kvsj\" (UniqueName: \"kubernetes.io/projected/7766c1c7-0747-4391-af89-7ea28f9d9fb6-kube-api-access-8kvsj\") pod \"kuadrant-operator-catalog-m56xq\" (UID: \"7766c1c7-0747-4391-af89-7ea28f9d9fb6\") " pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:44:51.074726 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:51.074617 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:44:51.217524 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:51.217488 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-m56xq"] Jun 08 16:44:51.259586 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:44:51.259555 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7766c1c7_0747_4391_af89_7ea28f9d9fb6.slice/crio-ccb6fcb13f59d301676f42cc8d7de65861b8908e16b41d557091862bdfd3af8a WatchSource:0}: Error finding container ccb6fcb13f59d301676f42cc8d7de65861b8908e16b41d557091862bdfd3af8a: Status 404 returned error can't find the container with id ccb6fcb13f59d301676f42cc8d7de65861b8908e16b41d557091862bdfd3af8a Jun 08 16:44:51.740423 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:51.740386 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" event={"ID":"7766c1c7-0747-4391-af89-7ea28f9d9fb6","Type":"ContainerStarted","Data":"ccb6fcb13f59d301676f42cc8d7de65861b8908e16b41d557091862bdfd3af8a"} Jun 08 16:44:53.755502 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:53.755467 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" event={"ID":"ca7d63df-0262-4500-b5cd-d42382363201","Type":"ContainerStarted","Data":"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9"} Jun 08 16:44:53.755987 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:53.755566 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" podUID="ca7d63df-0262-4500-b5cd-d42382363201" containerName="registry-server" containerID="cri-o://d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9" gracePeriod=2 Jun 08 16:44:53.757096 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:53.757071 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" event={"ID":"7766c1c7-0747-4391-af89-7ea28f9d9fb6","Type":"ContainerStarted","Data":"2bcb28565f42d1d44607aa4723ce94ad12598f8ac225e1ecbb1b39c94b5e6bd4"} Jun 08 16:44:53.797581 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:53.797531 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" podStartSLOduration=1.628714257 podStartE2EDuration="3.797514689s" totalImagesPullingTime="2.168800432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:44:53.775627337 +0000 UTC m=+698.161815735" watchObservedRunningTime="2026-06-08 16:44:53.797514689 +0000 UTC m=+698.183703083" Jun 08 16:44:53.797755 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:53.797612 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" podStartSLOduration=2.242624912 podStartE2EDuration="3.797607243s" totalImagesPullingTime="1.554982331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:44:53.795539388 +0000 UTC m=+698.181727782" watchObservedRunningTime="2026-06-08 16:44:53.797607243 +0000 UTC m=+698.183795638" Jun 08 16:44:54.006152 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.006086 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:54.061982 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.061946 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rr94\" (UniqueName: \"kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94\") pod \"ca7d63df-0262-4500-b5cd-d42382363201\" (UID: \"ca7d63df-0262-4500-b5cd-d42382363201\") " Jun 08 16:44:54.064345 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.064323 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94" (OuterVolumeSpecName: "kube-api-access-6rr94") pod "ca7d63df-0262-4500-b5cd-d42382363201" (UID: "ca7d63df-0262-4500-b5cd-d42382363201"). InnerVolumeSpecName "kube-api-access-6rr94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:44:54.162814 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.162774 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rr94\" (UniqueName: \"kubernetes.io/projected/ca7d63df-0262-4500-b5cd-d42382363201-kube-api-access-6rr94\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:44:54.762503 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.762466 2559 generic.go:356] "Generic (PLEG): container finished" podID="ca7d63df-0262-4500-b5cd-d42382363201" containerID="d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9" exitCode=0 Jun 08 16:44:54.762931 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.762511 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" Jun 08 16:44:54.762931 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.762552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" event={"ID":"ca7d63df-0262-4500-b5cd-d42382363201","Type":"ContainerDied","Data":"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9"} Jun 08 16:44:54.762931 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.762604 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9x7v" event={"ID":"ca7d63df-0262-4500-b5cd-d42382363201","Type":"ContainerDied","Data":"82d4e50019dfd9ae6961256759eeee2e8074f37371bd47ac74d8b9051e5d6d66"} Jun 08 16:44:54.762931 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.762625 2559 scope.go:117] "RemoveContainer" containerID="d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9" Jun 08 16:44:54.771559 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.771538 2559 scope.go:117] "RemoveContainer" containerID="d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9" Jun 08 16:44:54.771880 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:44:54.771861 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9\": container with ID starting with d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9 not found: ID does not exist" containerID="d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9" Jun 08 16:44:54.771957 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.771894 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9"} err="failed to get container status \"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9\": rpc error: code = NotFound desc = could not find container \"d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9\": container with ID starting with d2b4f4342ab8c4998dc045c9070d76cfecc569e6a0992667bb83db6bfad909d9 not found: ID does not exist" Jun 08 16:44:54.784197 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.784168 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:54.789581 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:54.789559 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9x7v"] Jun 08 16:44:56.118987 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:44:56.118948 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca7d63df-0262-4500-b5cd-d42382363201" path="/var/lib/kubelet/pods/ca7d63df-0262-4500-b5cd-d42382363201/volumes" Jun 08 16:45:01.075664 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:01.075591 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:45:01.076083 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:01.075696 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:45:01.098821 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:01.098790 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:45:01.821796 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:01.821765 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-m56xq" Jun 08 16:45:23.548113 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.548074 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks"] Jun 08 16:45:23.548549 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.548427 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca7d63df-0262-4500-b5cd-d42382363201" containerName="registry-server" Jun 08 16:45:23.548549 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.548440 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca7d63df-0262-4500-b5cd-d42382363201" containerName="registry-server" Jun 08 16:45:23.548549 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.548494 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca7d63df-0262-4500-b5cd-d42382363201" containerName="registry-server" Jun 08 16:45:23.552675 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.552655 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.555688 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.555662 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-72w7n\"" Jun 08 16:45:23.567099 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.567063 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks"] Jun 08 16:45:23.629176 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.629136 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqmv\" (UniqueName: \"kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.629385 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.629186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.730453 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.730410 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqmv\" (UniqueName: \"kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.730624 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.730471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.730905 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.730886 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.743077 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.743044 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqmv\" (UniqueName: \"kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:23.863100 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:23.863063 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:24.003564 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:24.003530 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks"] Jun 08 16:45:24.007347 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:45:24.007321 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241f1c33_a3e1_4644_ae39_583fac33bb13.slice/crio-908cbe33a46450120f8e0fd901ebbe9da276389e6c05fcdd5922673632329941 WatchSource:0}: Error finding container 908cbe33a46450120f8e0fd901ebbe9da276389e6c05fcdd5922673632329941: Status 404 returned error can't find the container with id 908cbe33a46450120f8e0fd901ebbe9da276389e6c05fcdd5922673632329941 Jun 08 16:45:24.905090 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:24.905057 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" event={"ID":"241f1c33-a3e1-4644-ae39-583fac33bb13","Type":"ContainerStarted","Data":"908cbe33a46450120f8e0fd901ebbe9da276389e6c05fcdd5922673632329941"} Jun 08 16:45:26.315138 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.315106 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rh996"] Jun 08 16:45:26.319939 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.319921 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:26.322780 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.322763 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-pkf6n\"" Jun 08 16:45:26.335744 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.335712 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rh996"] Jun 08 16:45:26.353826 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.353790 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m2l8\" (UniqueName: \"kubernetes.io/projected/285835d0-dbd4-4fff-a6fe-a7d1f954e297-kube-api-access-4m2l8\") pod \"authorino-operator-75cc8d5cb7-rh996\" (UID: \"285835d0-dbd4-4fff-a6fe-a7d1f954e297\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:26.455215 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.455171 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m2l8\" (UniqueName: \"kubernetes.io/projected/285835d0-dbd4-4fff-a6fe-a7d1f954e297-kube-api-access-4m2l8\") pod \"authorino-operator-75cc8d5cb7-rh996\" (UID: \"285835d0-dbd4-4fff-a6fe-a7d1f954e297\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:26.474566 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.474530 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m2l8\" (UniqueName: \"kubernetes.io/projected/285835d0-dbd4-4fff-a6fe-a7d1f954e297-kube-api-access-4m2l8\") pod \"authorino-operator-75cc8d5cb7-rh996\" (UID: \"285835d0-dbd4-4fff-a6fe-a7d1f954e297\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:26.630372 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.630332 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:26.799875 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.799848 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rh996"] Jun 08 16:45:26.800986 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:45:26.800958 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod285835d0_dbd4_4fff_a6fe_a7d1f954e297.slice/crio-4ccccafb06b626bb0f15a566932a31780f67cc97c755bf08acc3b18afaca726c WatchSource:0}: Error finding container 4ccccafb06b626bb0f15a566932a31780f67cc97c755bf08acc3b18afaca726c: Status 404 returned error can't find the container with id 4ccccafb06b626bb0f15a566932a31780f67cc97c755bf08acc3b18afaca726c Jun 08 16:45:26.917987 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:26.917887 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" event={"ID":"285835d0-dbd4-4fff-a6fe-a7d1f954e297","Type":"ContainerStarted","Data":"4ccccafb06b626bb0f15a566932a31780f67cc97c755bf08acc3b18afaca726c"} Jun 08 16:45:29.936478 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:29.936433 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" event={"ID":"285835d0-dbd4-4fff-a6fe-a7d1f954e297","Type":"ContainerStarted","Data":"317ef8d510146a0d98947299cec8c55d75eaea09a5ce50439bc8ae729509e6cd"} Jun 08 16:45:29.936957 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:29.936592 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:29.954755 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:29.954698 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" podStartSLOduration=1.62020195 podStartE2EDuration="3.954679214s" totalImagesPullingTime="2.334477264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:45:29.953449042 +0000 UTC m=+734.339637438" watchObservedRunningTime="2026-06-08 16:45:29.954679214 +0000 UTC m=+734.340867607" Jun 08 16:45:31.946736 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:31.946701 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" event={"ID":"241f1c33-a3e1-4644-ae39-583fac33bb13","Type":"ContainerStarted","Data":"10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336"} Jun 08 16:45:31.947162 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:31.946791 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:31.995992 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:31.995930 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" podStartSLOduration=1.9323639400000001 podStartE2EDuration="8.995910838s" totalImagesPullingTime="7.063546898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:45:31.991998452 +0000 UTC m=+736.378186848" watchObservedRunningTime="2026-06-08 16:45:31.995910838 +0000 UTC m=+736.382099232" Jun 08 16:45:40.943260 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:40.943227 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rh996" Jun 08 16:45:42.953794 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:42.953758 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:44.714765 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.714730 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks"] Jun 08 16:45:44.715155 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.714928 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" containerName="manager" containerID="cri-o://10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336" gracePeriod=2 Jun 08 16:45:44.722930 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.722902 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks"] Jun 08 16:45:44.767520 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.767485 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:45:44.768001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.767979 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" containerName="manager" Jun 08 16:45:44.768001 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.768001 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" containerName="manager" Jun 08 16:45:44.768138 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.768077 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" containerName="manager" Jun 08 16:45:44.771180 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.771159 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:44.792250 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.792218 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:45:44.849914 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:44.849876 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:44.940374 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.940267 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmfbf\" (UniqueName: \"kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:44.940374 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.940325 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:44.964831 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:44.964768 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:44.969775 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:44.969743 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:45.013978 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.013946 2559 generic.go:356] "Generic (PLEG): container finished" podID="241f1c33-a3e1-4644-ae39-583fac33bb13" containerID="10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336" exitCode=0 Jun 08 16:45:45.014127 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.014002 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:45.014127 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.014028 2559 scope.go:117] "RemoveContainer" containerID="10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336" Jun 08 16:45:45.018163 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:45.018126 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:45.022579 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.022560 2559 scope.go:117] "RemoveContainer" containerID="10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336" Jun 08 16:45:45.022871 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:45.022853 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336\": container with ID starting with 10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336 not found: ID does not exist" containerID="10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336" Jun 08 16:45:45.022920 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.022880 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336"} err="failed to get container status \"10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336\": rpc error: code = NotFound desc = could not find container \"10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336\": container with ID starting with 10a5779cd0c43d1c4ae0476e92965fe0f8eac8668d6bf36921aa088abafc1336 not found: ID does not exist" Jun 08 16:45:45.041691 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.041670 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfqmv\" (UniqueName: \"kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv\") pod \"241f1c33-a3e1-4644-ae39-583fac33bb13\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " Jun 08 16:45:45.041785 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.041709 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume\") pod \"241f1c33-a3e1-4644-ae39-583fac33bb13\" (UID: \"241f1c33-a3e1-4644-ae39-583fac33bb13\") " Jun 08 16:45:45.041833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.041821 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmfbf\" (UniqueName: \"kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:45.041878 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.041866 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:45.042212 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.042187 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:45.042323 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.042208 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "241f1c33-a3e1-4644-ae39-583fac33bb13" (UID: "241f1c33-a3e1-4644-ae39-583fac33bb13"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 16:45:45.043935 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.043914 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv" (OuterVolumeSpecName: "kube-api-access-wfqmv") pod "241f1c33-a3e1-4644-ae39-583fac33bb13" (UID: "241f1c33-a3e1-4644-ae39-583fac33bb13"). InnerVolumeSpecName "kube-api-access-wfqmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:45:45.052210 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.052186 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmfbf\" (UniqueName: \"kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lhtzd\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:45.104135 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.104104 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:45.142997 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.142960 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfqmv\" (UniqueName: \"kubernetes.io/projected/241f1c33-a3e1-4644-ae39-583fac33bb13-kube-api-access-wfqmv\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:45:45.142997 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.142992 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/241f1c33-a3e1-4644-ae39-583fac33bb13-extensions-socket-volume\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:45:45.246844 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:45.246817 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:45:45.248972 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:45:45.248940 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01cb7745_8a6b_4f42_b45a_8c52dfd55ffc.slice/crio-bc035a2f7d383822b1589ece08992406289cf810113a74500154d4f0de6f0b56 WatchSource:0}: Error finding container bc035a2f7d383822b1589ece08992406289cf810113a74500154d4f0de6f0b56: Status 404 returned error can't find the container with id bc035a2f7d383822b1589ece08992406289cf810113a74500154d4f0de6f0b56 Jun 08 16:45:45.325071 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:45.325038 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:46.020921 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:46.020887 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" event={"ID":"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc","Type":"ContainerStarted","Data":"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c"} Jun 08 16:45:46.021354 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:46.020937 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" event={"ID":"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc","Type":"ContainerStarted","Data":"bc035a2f7d383822b1589ece08992406289cf810113a74500154d4f0de6f0b56"} Jun 08 16:45:46.021354 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:46.021036 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:45:46.023737 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:46.023714 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:46.046869 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:46.046820 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" podStartSLOduration=2.046804242 podStartE2EDuration="2.046804242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:45:46.044036593 +0000 UTC m=+750.430224991" watchObservedRunningTime="2026-06-08 16:45:46.046804242 +0000 UTC m=+750.432992636" Jun 08 16:45:46.117474 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:45:46.117434 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kltks\" is forbidden: User \"system:node:ip-10-0-141-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-92.ec2.internal' and this object" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kltks" Jun 08 16:45:46.120428 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:46.120404 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241f1c33-a3e1-4644-ae39-583fac33bb13" path="/var/lib/kubelet/pods/241f1c33-a3e1-4644-ae39-583fac33bb13/volumes" Jun 08 16:45:57.027625 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:45:57.027592 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:46:16.338546 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.338496 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:46:16.339008 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.338790 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" podUID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" containerName="manager" containerID="cri-o://5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c" gracePeriod=10 Jun 08 16:46:16.598548 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.598521 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:46:16.734924 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.734834 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmfbf\" (UniqueName: \"kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf\") pod \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " Jun 08 16:46:16.735101 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.734958 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume\") pod \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\" (UID: \"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc\") " Jun 08 16:46:16.735361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.735335 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" (UID: "01cb7745-8a6b-4f42-b45a-8c52dfd55ffc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 16:46:16.737202 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.737179 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf" (OuterVolumeSpecName: "kube-api-access-tmfbf") pod "01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" (UID: "01cb7745-8a6b-4f42-b45a-8c52dfd55ffc"). InnerVolumeSpecName "kube-api-access-tmfbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:46:16.835909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.835873 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-extensions-socket-volume\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:46:16.835909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:16.835902 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmfbf\" (UniqueName: \"kubernetes.io/projected/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc-kube-api-access-tmfbf\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:46:17.166169 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.166133 2559 generic.go:356] "Generic (PLEG): container finished" podID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" containerID="5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c" exitCode=0 Jun 08 16:46:17.166348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.166179 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" Jun 08 16:46:17.166348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.166222 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" event={"ID":"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc","Type":"ContainerDied","Data":"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c"} Jun 08 16:46:17.166348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.166277 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd" event={"ID":"01cb7745-8a6b-4f42-b45a-8c52dfd55ffc","Type":"ContainerDied","Data":"bc035a2f7d383822b1589ece08992406289cf810113a74500154d4f0de6f0b56"} Jun 08 16:46:17.166348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.166304 2559 scope.go:117] "RemoveContainer" containerID="5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c" Jun 08 16:46:17.175581 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.175565 2559 scope.go:117] "RemoveContainer" containerID="5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c" Jun 08 16:46:17.175894 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:46:17.175876 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c\": container with ID starting with 5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c not found: ID does not exist" containerID="5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c" Jun 08 16:46:17.175952 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.175904 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c"} err="failed to get container status \"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c\": rpc error: code = NotFound desc = could not find container \"5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c\": container with ID starting with 5bf16300c6c7637a41169d6c660f3ba94b50c14de807748e3d1b18ae1676d07c not found: ID does not exist" Jun 08 16:46:17.203525 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.203492 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:46:17.215275 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:17.215247 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lhtzd"] Jun 08 16:46:18.117733 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:18.117702 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" path="/var/lib/kubelet/pods/01cb7745-8a6b-4f42-b45a-8c52dfd55ffc/volumes" Jun 08 16:46:32.603754 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.603674 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t"] Jun 08 16:46:32.604199 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.604159 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" containerName="manager" Jun 08 16:46:32.604199 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.604177 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" containerName="manager" Jun 08 16:46:32.604294 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.604258 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="01cb7745-8a6b-4f42-b45a-8c52dfd55ffc" containerName="manager" Jun 08 16:46:32.608535 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.608515 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.611358 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.611341 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-62vmf\"" Jun 08 16:46:32.620855 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.620833 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t"] Jun 08 16:46:32.779221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779177 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779222 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779272 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779340 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779427 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779458 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779477 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.779739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.779541 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2hp\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-kube-api-access-6r2hp\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880288 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880328 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880353 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880369 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880452 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880491 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880820 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880790 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880872 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880836 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880872 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880855 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880984 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880858 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.880984 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.880931 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2hp\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-kube-api-access-6r2hp\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.881079 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.881024 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.881415 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.881397 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.882942 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.882920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.883135 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.883116 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.896355 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.896323 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.896461 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.896369 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2hp\" (UniqueName: \"kubernetes.io/projected/1bb94220-a0fa-422c-a75f-cb2cf39cc533-kube-api-access-6r2hp\") pod \"maas-default-gateway-openshift-default-687ff6996-h2j8t\" (UID: \"1bb94220-a0fa-422c-a75f-cb2cf39cc533\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:32.919542 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:32.919500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:33.060692 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.060663 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t"] Jun 08 16:46:33.063245 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:46:33.063220 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb94220_a0fa_422c_a75f_cb2cf39cc533.slice/crio-066274b58470f843b69bd72a0737b15878b27bc3e9ba6ac3f634f8f09a793500 WatchSource:0}: Error finding container 066274b58470f843b69bd72a0737b15878b27bc3e9ba6ac3f634f8f09a793500: Status 404 returned error can't find the container with id 066274b58470f843b69bd72a0737b15878b27bc3e9ba6ac3f634f8f09a793500 Jun 08 16:46:33.065292 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.065259 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:46:33.065366 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.065336 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:46:33.065407 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.065369 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:46:33.247447 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.247347 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" event={"ID":"1bb94220-a0fa-422c-a75f-cb2cf39cc533","Type":"ContainerStarted","Data":"04865424e5e3af535e3f15c4f2d54299a326a0755fc37ef7ea871b70ad28e432"} Jun 08 16:46:33.247447 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.247418 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" event={"ID":"1bb94220-a0fa-422c-a75f-cb2cf39cc533","Type":"ContainerStarted","Data":"066274b58470f843b69bd72a0737b15878b27bc3e9ba6ac3f634f8f09a793500"} Jun 08 16:46:33.268789 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.268724 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" podStartSLOduration=1.2687013409999999 podStartE2EDuration="1.268701341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:46:33.267416416 +0000 UTC m=+797.653604811" watchObservedRunningTime="2026-06-08 16:46:33.268701341 +0000 UTC m=+797.654889736" Jun 08 16:46:33.920585 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:33.920544 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:34.924812 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:34.924784 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:35.256814 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:35.256731 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:35.257930 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:35.257912 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:46:37.746035 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.745950 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:37.749329 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.749302 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.752788 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.752769 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d8bqx\"" Jun 08 16:46:37.752883 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.752773 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 08 16:46:37.757323 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.757289 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:37.823307 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.823276 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.823469 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.823316 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv58v\" (UniqueName: \"kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.840299 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.840251 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:37.923848 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.923808 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.923848 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.923859 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv58v\" (UniqueName: \"kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.924543 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.924515 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:37.933478 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:37.933438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv58v\" (UniqueName: \"kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v\") pod \"limitador-limitador-64c94fd64f-s5s8z\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:38.061048 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.060951 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:38.200835 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.200812 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:38.202840 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:46:38.202807 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92174cd_aec7_4ee6_8738_2097b5d0dd34.slice/crio-7c88efdd45c8e6e18c5afc377ce5c3a3f6611c1c93ab80a0217953c721df4e11 WatchSource:0}: Error finding container 7c88efdd45c8e6e18c5afc377ce5c3a3f6611c1c93ab80a0217953c721df4e11: Status 404 returned error can't find the container with id 7c88efdd45c8e6e18c5afc377ce5c3a3f6611c1c93ab80a0217953c721df4e11 Jun 08 16:46:38.272045 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.272007 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" event={"ID":"c92174cd-aec7-4ee6-8738-2097b5d0dd34","Type":"ContainerStarted","Data":"7c88efdd45c8e6e18c5afc377ce5c3a3f6611c1c93ab80a0217953c721df4e11"} Jun 08 16:46:38.764233 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.764193 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:46:38.769168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.769143 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:46:38.772025 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.771998 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z4l9v\"" Jun 08 16:46:38.773570 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.773540 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:46:38.830447 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.830415 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfr7\" (UniqueName: \"kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7\") pod \"authorino-75d556bd77-thq5f\" (UID: \"a1b55dec-994d-4bed-aee4-ffc9e8234412\") " pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:46:38.932432 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.931921 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfr7\" (UniqueName: \"kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7\") pod \"authorino-75d556bd77-thq5f\" (UID: \"a1b55dec-994d-4bed-aee4-ffc9e8234412\") " pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:46:38.942989 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:38.942920 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfr7\" (UniqueName: \"kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7\") pod \"authorino-75d556bd77-thq5f\" (UID: \"a1b55dec-994d-4bed-aee4-ffc9e8234412\") " pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:46:39.081395 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:39.081119 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:46:39.262433 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:39.262400 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:46:39.266340 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:46:39.266300 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1b55dec_994d_4bed_aee4_ffc9e8234412.slice/crio-5520840cdcf5ccee71160c482d2a80c7e788a32946d506aaf9af814cac3fc540 WatchSource:0}: Error finding container 5520840cdcf5ccee71160c482d2a80c7e788a32946d506aaf9af814cac3fc540: Status 404 returned error can't find the container with id 5520840cdcf5ccee71160c482d2a80c7e788a32946d506aaf9af814cac3fc540 Jun 08 16:46:39.279182 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:39.279140 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-thq5f" event={"ID":"a1b55dec-994d-4bed-aee4-ffc9e8234412","Type":"ContainerStarted","Data":"5520840cdcf5ccee71160c482d2a80c7e788a32946d506aaf9af814cac3fc540"} Jun 08 16:46:43.305206 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:43.305097 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-thq5f" event={"ID":"a1b55dec-994d-4bed-aee4-ffc9e8234412","Type":"ContainerStarted","Data":"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d"} Jun 08 16:46:43.307508 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:43.307466 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" event={"ID":"c92174cd-aec7-4ee6-8738-2097b5d0dd34","Type":"ContainerStarted","Data":"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09"} Jun 08 16:46:43.307668 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:43.307619 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:43.323786 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:43.323728 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-thq5f" podStartSLOduration=1.6931267779999999 podStartE2EDuration="5.323707675s" totalImagesPullingTime="3.630580897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:46:43.321920573 +0000 UTC m=+807.708108967" watchObservedRunningTime="2026-06-08 16:46:43.323707675 +0000 UTC m=+807.709896070" Jun 08 16:46:43.340368 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:43.340311 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" podStartSLOduration=1.693153867 podStartE2EDuration="6.34029514s" totalImagesPullingTime="4.647141273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:46:43.339223921 +0000 UTC m=+807.725412316" watchObservedRunningTime="2026-06-08 16:46:43.34029514 +0000 UTC m=+807.726483533" Jun 08 16:46:52.206568 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.206534 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:52.207193 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.206785 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" podUID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" containerName="limitador" containerID="cri-o://68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09" gracePeriod=30 Jun 08 16:46:52.207576 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.207496 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:52.758673 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.758620 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:52.860427 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.860396 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv58v\" (UniqueName: \"kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v\") pod \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " Jun 08 16:46:52.860588 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.860451 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file\") pod \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\" (UID: \"c92174cd-aec7-4ee6-8738-2097b5d0dd34\") " Jun 08 16:46:52.860856 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.860827 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file" (OuterVolumeSpecName: "config-file") pod "c92174cd-aec7-4ee6-8738-2097b5d0dd34" (UID: "c92174cd-aec7-4ee6-8738-2097b5d0dd34"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:46:52.862606 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.862582 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v" (OuterVolumeSpecName: "kube-api-access-lv58v") pod "c92174cd-aec7-4ee6-8738-2097b5d0dd34" (UID: "c92174cd-aec7-4ee6-8738-2097b5d0dd34"). InnerVolumeSpecName "kube-api-access-lv58v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:46:52.961537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.961495 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv58v\" (UniqueName: \"kubernetes.io/projected/c92174cd-aec7-4ee6-8738-2097b5d0dd34-kube-api-access-lv58v\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:46:52.961537 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:52.961532 2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c92174cd-aec7-4ee6-8738-2097b5d0dd34-config-file\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:46:53.356425 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.356388 2559 generic.go:356] "Generic (PLEG): container finished" podID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" containerID="68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09" exitCode=0 Jun 08 16:46:53.356909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.356447 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" Jun 08 16:46:53.356909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.356476 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" event={"ID":"c92174cd-aec7-4ee6-8738-2097b5d0dd34","Type":"ContainerDied","Data":"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09"} Jun 08 16:46:53.356909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.356530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s5s8z" event={"ID":"c92174cd-aec7-4ee6-8738-2097b5d0dd34","Type":"ContainerDied","Data":"7c88efdd45c8e6e18c5afc377ce5c3a3f6611c1c93ab80a0217953c721df4e11"} Jun 08 16:46:53.356909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.356553 2559 scope.go:117] "RemoveContainer" containerID="68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09" Jun 08 16:46:53.366929 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.366908 2559 scope.go:117] "RemoveContainer" containerID="68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09" Jun 08 16:46:53.367285 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:46:53.367262 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09\": container with ID starting with 68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09 not found: ID does not exist" containerID="68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09" Jun 08 16:46:53.367335 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.367298 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09"} err="failed to get container status \"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09\": rpc error: code = NotFound desc = could not find container \"68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09\": container with ID starting with 68f6a887da7b55fa90ae1981552fd69ebf59e5c1ef7232c04b2ac1955813ba09 not found: ID does not exist" Jun 08 16:46:53.382352 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.382315 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:53.385722 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.385692 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s5s8z"] Jun 08 16:46:53.875216 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.875172 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-424jq"] Jun 08 16:46:53.875545 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.875532 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" containerName="limitador" Jun 08 16:46:53.875545 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.875545 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" containerName="limitador" Jun 08 16:46:53.875697 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.875618 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" containerName="limitador" Jun 08 16:46:53.879957 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.879922 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:53.882907 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.882883 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 08 16:46:53.888652 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.888612 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-424jq"] Jun 08 16:46:53.971036 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.970991 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/67fd4edf-95bd-4180-961d-8671251665e8-data\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:53.971221 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:53.971058 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdgf\" (UniqueName: \"kubernetes.io/projected/67fd4edf-95bd-4180-961d-8671251665e8-kube-api-access-rrdgf\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.071807 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.071766 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdgf\" (UniqueName: \"kubernetes.io/projected/67fd4edf-95bd-4180-961d-8671251665e8-kube-api-access-rrdgf\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.071989 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.071879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/67fd4edf-95bd-4180-961d-8671251665e8-data\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.072207 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.072191 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/67fd4edf-95bd-4180-961d-8671251665e8-data\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.081287 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.081262 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdgf\" (UniqueName: \"kubernetes.io/projected/67fd4edf-95bd-4180-961d-8671251665e8-kube-api-access-rrdgf\") pod \"postgres-69fb4594cf-424jq\" (UID: \"67fd4edf-95bd-4180-961d-8671251665e8\") " pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.117917 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.117882 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c92174cd-aec7-4ee6-8738-2097b5d0dd34" path="/var/lib/kubelet/pods/c92174cd-aec7-4ee6-8738-2097b5d0dd34/volumes" Jun 08 16:46:54.192295 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.192207 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:54.328751 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.328724 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-424jq"] Jun 08 16:46:54.330452 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:46:54.330426 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fd4edf_95bd_4180_961d_8671251665e8.slice/crio-1fb2d41aafb77f21db35a31dab51bd1b74b30fb6e259da4f9a12c7b2aa76861a WatchSource:0}: Error finding container 1fb2d41aafb77f21db35a31dab51bd1b74b30fb6e259da4f9a12c7b2aa76861a: Status 404 returned error can't find the container with id 1fb2d41aafb77f21db35a31dab51bd1b74b30fb6e259da4f9a12c7b2aa76861a Jun 08 16:46:54.362736 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:54.362697 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-424jq" event={"ID":"67fd4edf-95bd-4180-961d-8671251665e8","Type":"ContainerStarted","Data":"1fb2d41aafb77f21db35a31dab51bd1b74b30fb6e259da4f9a12c7b2aa76861a"} Jun 08 16:46:57.380265 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:57.380225 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-424jq" event={"ID":"67fd4edf-95bd-4180-961d-8671251665e8","Type":"ContainerStarted","Data":"8f0995b498b7377223c1c500a55533f41dacc15fa7f4784649f512fffbc739f6"} Jun 08 16:46:57.380692 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:57.380335 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:46:57.399240 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:46:57.399182 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-424jq" podStartSLOduration=2.078207681 podStartE2EDuration="4.399164255s" totalImagesPullingTime="2.320956574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:46:57.39708177 +0000 UTC m=+821.783270163" watchObservedRunningTime="2026-06-08 16:46:57.399164255 +0000 UTC m=+821.785352643" Jun 08 16:47:03.422122 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:03.422093 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-424jq" Jun 08 16:47:12.698900 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.698860 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-hljr7"] Jun 08 16:47:12.705364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.705343 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" Jun 08 16:47:12.710212 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.710189 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 16:47:12.711779 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.711760 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-4997s\"" Jun 08 16:47:12.711885 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.711811 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 16:47:12.718330 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.718305 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-hljr7"] Jun 08 16:47:12.853954 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.853916 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xhpj\" (UniqueName: \"kubernetes.io/projected/3f457d76-c1bd-4dce-9665-fcc935d7b283-kube-api-access-5xhpj\") pod \"keycloak-operator-d67dbbb48-hljr7\" (UID: \"3f457d76-c1bd-4dce-9665-fcc935d7b283\") " pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" Jun 08 16:47:12.955443 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.955348 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xhpj\" (UniqueName: \"kubernetes.io/projected/3f457d76-c1bd-4dce-9665-fcc935d7b283-kube-api-access-5xhpj\") pod \"keycloak-operator-d67dbbb48-hljr7\" (UID: \"3f457d76-c1bd-4dce-9665-fcc935d7b283\") " pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" Jun 08 16:47:12.964211 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:12.964175 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xhpj\" (UniqueName: \"kubernetes.io/projected/3f457d76-c1bd-4dce-9665-fcc935d7b283-kube-api-access-5xhpj\") pod \"keycloak-operator-d67dbbb48-hljr7\" (UID: \"3f457d76-c1bd-4dce-9665-fcc935d7b283\") " pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" Jun 08 16:47:13.016013 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:13.015978 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" Jun 08 16:47:13.156290 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:13.156261 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-hljr7"] Jun 08 16:47:13.158439 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:47:13.158409 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f457d76_c1bd_4dce_9665_fcc935d7b283.slice/crio-3369624f9af86a83acc42886a97130e10a8912e3a314040342a9118a55fb7ca1 WatchSource:0}: Error finding container 3369624f9af86a83acc42886a97130e10a8912e3a314040342a9118a55fb7ca1: Status 404 returned error can't find the container with id 3369624f9af86a83acc42886a97130e10a8912e3a314040342a9118a55fb7ca1 Jun 08 16:47:13.459888 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:13.459850 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" event={"ID":"3f457d76-c1bd-4dce-9665-fcc935d7b283","Type":"ContainerStarted","Data":"3369624f9af86a83acc42886a97130e10a8912e3a314040342a9118a55fb7ca1"} Jun 08 16:47:20.504135 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:20.504090 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" event={"ID":"3f457d76-c1bd-4dce-9665-fcc935d7b283","Type":"ContainerStarted","Data":"408ebb53f4367bbd8efed3ff0536e985a913cba79982434ff6f55b69cf80db5e"} Jun 08 16:47:20.521465 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:20.521384 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-hljr7" podStartSLOduration=1.838677761 podStartE2EDuration="8.521364375s" totalImagesPullingTime="6.682686614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:47:20.5208165 +0000 UTC m=+844.907004933" watchObservedRunningTime="2026-06-08 16:47:20.521364375 +0000 UTC m=+844.907552750" Jun 08 16:47:58.772463 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:58.772424 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tf84t"] Jun 08 16:47:58.782913 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:58.782884 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:58.783109 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:58.783051 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tf84t"] Jun 08 16:47:58.972553 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:58.972513 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fjbn\" (UniqueName: \"kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn\") pod \"authorino-5d4b67c446-tf84t\" (UID: \"092d6eed-d6d5-4a3c-b425-d73eee96e849\") " pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:59.074028 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.073932 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fjbn\" (UniqueName: \"kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn\") pod \"authorino-5d4b67c446-tf84t\" (UID: \"092d6eed-d6d5-4a3c-b425-d73eee96e849\") " pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:59.085102 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.085072 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tf84t"] Jun 08 16:47:59.085373 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:47:59.085353 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6fjbn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tf84t" podUID="092d6eed-d6d5-4a3c-b425-d73eee96e849" Jun 08 16:47:59.088461 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.088438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fjbn\" (UniqueName: \"kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn\") pod \"authorino-5d4b67c446-tf84t\" (UID: \"092d6eed-d6d5-4a3c-b425-d73eee96e849\") " pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:59.133853 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.133812 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:47:59.137397 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.137376 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:47:59.145337 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.145302 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:47:59.276836 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.276789 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkv8g\" (UniqueName: \"kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g\") pod \"authorino-7fd47dc7f8-5dwdf\" (UID: \"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b\") " pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:47:59.377569 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.377529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkv8g\" (UniqueName: \"kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g\") pod \"authorino-7fd47dc7f8-5dwdf\" (UID: \"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b\") " pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:47:59.389052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.389020 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkv8g\" (UniqueName: \"kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g\") pod \"authorino-7fd47dc7f8-5dwdf\" (UID: \"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b\") " pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:47:59.409748 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.409710 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:47:59.410021 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.410008 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:47:59.548560 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.548536 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:47:59.550471 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:47:59.550445 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12d4913b_ac3f_4bc6_acfc_bb6c0ee7499b.slice/crio-efeab4cb88972a745483b6ff60f8ef27ac60bd87c90e0f2f8d7e1e7a36cc6140 WatchSource:0}: Error finding container efeab4cb88972a745483b6ff60f8ef27ac60bd87c90e0f2f8d7e1e7a36cc6140: Status 404 returned error can't find the container with id efeab4cb88972a745483b6ff60f8ef27ac60bd87c90e0f2f8d7e1e7a36cc6140 Jun 08 16:47:59.697293 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.697205 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" event={"ID":"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b","Type":"ContainerStarted","Data":"efeab4cb88972a745483b6ff60f8ef27ac60bd87c90e0f2f8d7e1e7a36cc6140"} Jun 08 16:47:59.697293 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.697256 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:59.705941 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.705915 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:47:59.881345 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.881305 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fjbn\" (UniqueName: \"kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn\") pod \"092d6eed-d6d5-4a3c-b425-d73eee96e849\" (UID: \"092d6eed-d6d5-4a3c-b425-d73eee96e849\") " Jun 08 16:47:59.883824 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.883789 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn" (OuterVolumeSpecName: "kube-api-access-6fjbn") pod "092d6eed-d6d5-4a3c-b425-d73eee96e849" (UID: "092d6eed-d6d5-4a3c-b425-d73eee96e849"). InnerVolumeSpecName "kube-api-access-6fjbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:47:59.983025 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:47:59.982999 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fjbn\" (UniqueName: \"kubernetes.io/projected/092d6eed-d6d5-4a3c-b425-d73eee96e849-kube-api-access-6fjbn\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:48:00.703507 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.703474 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tf84t" Jun 08 16:48:00.703742 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.703517 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" event={"ID":"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b","Type":"ContainerStarted","Data":"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690"} Jun 08 16:48:00.703742 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.703536 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" podUID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" containerName="authorino" containerID="cri-o://b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690" gracePeriod=30 Jun 08 16:48:00.723862 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.723818 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" podStartSLOduration=1.3143904499999999 podStartE2EDuration="1.723802673s" totalImagesPullingTime="409.412223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:47:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:48:00.722678583 +0000 UTC m=+885.108866983" watchObservedRunningTime="2026-06-08 16:48:00.723802673 +0000 UTC m=+885.109991101" Jun 08 16:48:00.745963 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.745929 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tf84t"] Jun 08 16:48:00.749825 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.749796 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tf84t"] Jun 08 16:48:00.965051 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:00.964990 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:48:01.029686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.024120 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:48:01.029686 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.024373 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-thq5f" podUID="a1b55dec-994d-4bed-aee4-ffc9e8234412" containerName="authorino" containerID="cri-o://d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d" gracePeriod=30 Jun 08 16:48:01.092822 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.092788 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkv8g\" (UniqueName: \"kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g\") pod \"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b\" (UID: \"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b\") " Jun 08 16:48:01.095048 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.095011 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g" (OuterVolumeSpecName: "kube-api-access-qkv8g") pod "12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" (UID: "12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b"). InnerVolumeSpecName "kube-api-access-qkv8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:48:01.193984 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.193945 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkv8g\" (UniqueName: \"kubernetes.io/projected/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b-kube-api-access-qkv8g\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:48:01.294417 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.294389 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:48:01.395625 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.395588 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfr7\" (UniqueName: \"kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7\") pod \"a1b55dec-994d-4bed-aee4-ffc9e8234412\" (UID: \"a1b55dec-994d-4bed-aee4-ffc9e8234412\") " Jun 08 16:48:01.397820 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.397796 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7" (OuterVolumeSpecName: "kube-api-access-lkfr7") pod "a1b55dec-994d-4bed-aee4-ffc9e8234412" (UID: "a1b55dec-994d-4bed-aee4-ffc9e8234412"). InnerVolumeSpecName "kube-api-access-lkfr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:48:01.496553 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.496516 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfr7\" (UniqueName: \"kubernetes.io/projected/a1b55dec-994d-4bed-aee4-ffc9e8234412-kube-api-access-lkfr7\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:48:01.709187 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.709151 2559 generic.go:356] "Generic (PLEG): container finished" podID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" containerID="b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690" exitCode=0 Jun 08 16:48:01.709394 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.709209 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" Jun 08 16:48:01.709394 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.709245 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" event={"ID":"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b","Type":"ContainerDied","Data":"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690"} Jun 08 16:48:01.709394 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.709298 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fd47dc7f8-5dwdf" event={"ID":"12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b","Type":"ContainerDied","Data":"efeab4cb88972a745483b6ff60f8ef27ac60bd87c90e0f2f8d7e1e7a36cc6140"} Jun 08 16:48:01.709394 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.709331 2559 scope.go:117] "RemoveContainer" containerID="b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690" Jun 08 16:48:01.710694 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.710649 2559 generic.go:356] "Generic (PLEG): container finished" podID="a1b55dec-994d-4bed-aee4-ffc9e8234412" containerID="d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d" exitCode=0 Jun 08 16:48:01.710767 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.710693 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-thq5f" Jun 08 16:48:01.710767 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.710736 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-thq5f" event={"ID":"a1b55dec-994d-4bed-aee4-ffc9e8234412","Type":"ContainerDied","Data":"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d"} Jun 08 16:48:01.710860 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.710781 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-thq5f" event={"ID":"a1b55dec-994d-4bed-aee4-ffc9e8234412","Type":"ContainerDied","Data":"5520840cdcf5ccee71160c482d2a80c7e788a32946d506aaf9af814cac3fc540"} Jun 08 16:48:01.720232 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.720211 2559 scope.go:117] "RemoveContainer" containerID="b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690" Jun 08 16:48:01.720550 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:48:01.720531 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690\": container with ID starting with b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690 not found: ID does not exist" containerID="b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690" Jun 08 16:48:01.720590 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.720559 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690"} err="failed to get container status \"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690\": rpc error: code = NotFound desc = could not find container \"b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690\": container with ID starting with b4bcdb40925270969d103ff4399f685caa4cc71ad4f6a2fcf1b587ccdf55f690 not found: ID does not exist" Jun 08 16:48:01.720590 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.720577 2559 scope.go:117] "RemoveContainer" containerID="d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d" Jun 08 16:48:01.728781 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.728765 2559 scope.go:117] "RemoveContainer" containerID="d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d" Jun 08 16:48:01.729051 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:48:01.729034 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d\": container with ID starting with d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d not found: ID does not exist" containerID="d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d" Jun 08 16:48:01.729099 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.729058 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d"} err="failed to get container status \"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d\": rpc error: code = NotFound desc = could not find container \"d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d\": container with ID starting with d84e4bc21df0048044ced0f65e8a2028fc3fa30f1ed5501ad290d9782451530d not found: ID does not exist" Jun 08 16:48:01.743288 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.743244 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:48:01.748796 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.748763 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-thq5f"] Jun 08 16:48:01.759405 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.759375 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:48:01.763540 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:01.763512 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fd47dc7f8-5dwdf"] Jun 08 16:48:02.118235 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:02.118201 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="092d6eed-d6d5-4a3c-b425-d73eee96e849" path="/var/lib/kubelet/pods/092d6eed-d6d5-4a3c-b425-d73eee96e849/volumes" Jun 08 16:48:02.118607 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:02.118420 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" path="/var/lib/kubelet/pods/12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b/volumes" Jun 08 16:48:02.118720 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:02.118707 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b55dec-994d-4bed-aee4-ffc9e8234412" path="/var/lib/kubelet/pods/a1b55dec-994d-4bed-aee4-ffc9e8234412/volumes" Jun 08 16:48:04.838308 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838273 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:48:04.838803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838717 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" containerName="authorino" Jun 08 16:48:04.838803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838731 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" containerName="authorino" Jun 08 16:48:04.838803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838747 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b55dec-994d-4bed-aee4-ffc9e8234412" containerName="authorino" Jun 08 16:48:04.838803 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838756 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b55dec-994d-4bed-aee4-ffc9e8234412" containerName="authorino" Jun 08 16:48:04.838961 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838829 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="12d4913b-ac3f-4bc6-acfc-bb6c0ee7499b" containerName="authorino" Jun 08 16:48:04.838961 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.838843 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b55dec-994d-4bed-aee4-ffc9e8234412" containerName="authorino" Jun 08 16:48:04.843584 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.843555 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:04.846872 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.846846 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 16:48:04.846982 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.846875 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 16:48:04.846982 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.846850 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qgwwh\"" Jun 08 16:48:04.853711 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.853681 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:48:04.924093 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.924056 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxsjw\" (UniqueName: \"kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:04.924284 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:04.924103 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.024999 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.024966 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxsjw\" (UniqueName: \"kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.024999 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.025012 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.025228 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:48:05.025161 2559 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 08 16:48:05.025228 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:48:05.025216 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert podName:490403e1-688e-4cea-8233-14bcc0cf6d02 nodeName:}" failed. No retries permitted until 2026-06-08 16:48:05.525199371 +0000 UTC m=+889.911387743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert") pod "maas-controller-76f9db8d84-7hmbq" (UID: "490403e1-688e-4cea-8233-14bcc0cf6d02") : secret "maas-controller-webhook-cert" not found Jun 08 16:48:05.036925 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.036891 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxsjw\" (UniqueName: \"kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.528760 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.528719 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.531272 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.531252 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") pod \"maas-controller-76f9db8d84-7hmbq\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.754969 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.754925 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:05.901102 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:05.901077 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:48:05.903590 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:48:05.903430 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490403e1_688e_4cea_8233_14bcc0cf6d02.slice/crio-47b74607be11fb8a84e80aa0c5586232620f0d825acfe019e42e8f377489b186 WatchSource:0}: Error finding container 47b74607be11fb8a84e80aa0c5586232620f0d825acfe019e42e8f377489b186: Status 404 returned error can't find the container with id 47b74607be11fb8a84e80aa0c5586232620f0d825acfe019e42e8f377489b186 Jun 08 16:48:06.739395 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:06.739357 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" event={"ID":"490403e1-688e-4cea-8233-14bcc0cf6d02","Type":"ContainerStarted","Data":"47b74607be11fb8a84e80aa0c5586232620f0d825acfe019e42e8f377489b186"} Jun 08 16:48:08.751489 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:08.751448 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" event={"ID":"490403e1-688e-4cea-8233-14bcc0cf6d02","Type":"ContainerStarted","Data":"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff"} Jun 08 16:48:08.752020 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:08.751584 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:08.771887 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:08.771834 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" podStartSLOduration=2.417575895 podStartE2EDuration="4.771816793s" totalImagesPullingTime="2.354240898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:48:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:48:08.769405731 +0000 UTC m=+893.155594124" watchObservedRunningTime="2026-06-08 16:48:08.771816793 +0000 UTC m=+893.158005186" Jun 08 16:48:16.119181 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:16.119149 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:48:16.119959 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:16.119942 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:48:19.764397 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:19.764362 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:48:39.201052 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.201018 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-x72c7"] Jun 08 16:48:39.204958 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.204926 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:39.207771 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.207744 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qt7fc\"" Jun 08 16:48:39.207989 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.207971 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 16:48:39.214277 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.214248 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-x72c7"] Jun 08 16:48:39.319530 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.319489 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gcg\" (UniqueName: \"kubernetes.io/projected/ca2483f0-0110-40c0-b699-13404f7be2e0-kube-api-access-n2gcg\") pod \"payload-pre-processing-67d7689b65-x72c7\" (UID: \"ca2483f0-0110-40c0-b699-13404f7be2e0\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:39.420091 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.420054 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gcg\" (UniqueName: \"kubernetes.io/projected/ca2483f0-0110-40c0-b699-13404f7be2e0-kube-api-access-n2gcg\") pod \"payload-pre-processing-67d7689b65-x72c7\" (UID: \"ca2483f0-0110-40c0-b699-13404f7be2e0\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:39.430827 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.430796 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gcg\" (UniqueName: \"kubernetes.io/projected/ca2483f0-0110-40c0-b699-13404f7be2e0-kube-api-access-n2gcg\") pod \"payload-pre-processing-67d7689b65-x72c7\" (UID: \"ca2483f0-0110-40c0-b699-13404f7be2e0\") " pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:39.516909 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.516803 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:39.660472 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.660444 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-67d7689b65-x72c7"] Jun 08 16:48:39.662515 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:48:39.662470 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2483f0_0110_40c0_b699_13404f7be2e0.slice/crio-0d5b6fabcbb2220e1df7352cf5c0fbc3c626aeae2452483dca33b08aac476cf6 WatchSource:0}: Error finding container 0d5b6fabcbb2220e1df7352cf5c0fbc3c626aeae2452483dca33b08aac476cf6: Status 404 returned error can't find the container with id 0d5b6fabcbb2220e1df7352cf5c0fbc3c626aeae2452483dca33b08aac476cf6 Jun 08 16:48:39.903744 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:39.903660 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" event={"ID":"ca2483f0-0110-40c0-b699-13404f7be2e0","Type":"ContainerStarted","Data":"0d5b6fabcbb2220e1df7352cf5c0fbc3c626aeae2452483dca33b08aac476cf6"} Jun 08 16:48:41.928364 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:41.928320 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" event={"ID":"ca2483f0-0110-40c0-b699-13404f7be2e0","Type":"ContainerStarted","Data":"2599d532dcb5f8a4aa03d5afb6c758a9b03eb5c37f64cb0de9c180bf89c2c9f7"} Jun 08 16:48:41.928903 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:41.928471 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:48:41.946344 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:41.946290 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" podStartSLOduration=1.684612126 podStartE2EDuration="2.946273328s" totalImagesPullingTime="1.261661202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:48:41.944675375 +0000 UTC m=+926.330863770" watchObservedRunningTime="2026-06-08 16:48:41.946273328 +0000 UTC m=+926.332461722" Jun 08 16:48:46.954701 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.954606 2559 generic.go:356] "Generic (PLEG): container finished" podID="1bb94220-a0fa-422c-a75f-cb2cf39cc533" containerID="04865424e5e3af535e3f15c4f2d54299a326a0755fc37ef7ea871b70ad28e432" exitCode=0 Jun 08 16:48:46.954701 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.954683 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" event={"ID":"1bb94220-a0fa-422c-a75f-cb2cf39cc533","Type":"ContainerDied","Data":"04865424e5e3af535e3f15c4f2d54299a326a0755fc37ef7ea871b70ad28e432"} Jun 08 16:48:46.955126 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.955091 2559 scope.go:117] "RemoveContainer" containerID="04865424e5e3af535e3f15c4f2d54299a326a0755fc37ef7ea871b70ad28e432" Jun 08 16:48:46.956142 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.956106 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:48:46.956216 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.956198 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:48:46.956275 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:46.956259 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 16:48:47.961393 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:47.961362 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" event={"ID":"1bb94220-a0fa-422c-a75f-cb2cf39cc533","Type":"ContainerStarted","Data":"dba77732904ff20199ea781188fbfed1951346cbe27e7860524d4d92b711f41c"} Jun 08 16:48:48.920341 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:48.920298 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:48:48.921479 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:48.921451 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-h2j8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jun 08 16:48:48.921668 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:48.921505 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" podUID="1bb94220-a0fa-422c-a75f-cb2cf39cc533" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jun 08 16:48:49.920210 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:49.920166 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-h2j8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jun 08 16:48:49.920711 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:49.920232 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" podUID="1bb94220-a0fa-422c-a75f-cb2cf39cc533" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jun 08 16:48:50.257706 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:50.257596 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:48:50.920565 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:50.920524 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-h2j8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jun 08 16:48:50.921002 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:50.920586 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" podUID="1bb94220-a0fa-422c-a75f-cb2cf39cc533" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jun 08 16:48:51.937522 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:51.937482 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:48:51.939149 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:51.939124 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-h2j8t" Jun 08 16:48:52.935765 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:48:52.935726 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-67d7689b65-x72c7" Jun 08 16:49:27.478844 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.478809 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:49:27.484330 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.484310 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.488709 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.488684 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-z4l9v\"" Jun 08 16:49:27.488873 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.488685 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 08 16:49:27.488873 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.488696 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 08 16:49:27.494508 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.494481 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:49:27.572575 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.572533 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.572575 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.572574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.572903 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.572686 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2x8n\" (UniqueName: \"kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.673417 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.673377 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.673417 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.673424 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.673623 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.673463 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2x8n\" (UniqueName: \"kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.674125 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.674099 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.676215 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.676181 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.682334 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.682308 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2x8n\" (UniqueName: \"kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n\") pod \"authorino-fd874b6b8-zh4pc\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.794068 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.793980 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:49:27.932967 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:27.932942 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:49:27.935254 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:49:27.935227 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2489ac05_ea97_4034_aefa_4f87450babbf.slice/crio-6b9fc909dfd5fdbb76efe43b147a82154e549d89122362348a1e11a7bd0bdbd7 WatchSource:0}: Error finding container 6b9fc909dfd5fdbb76efe43b147a82154e549d89122362348a1e11a7bd0bdbd7: Status 404 returned error can't find the container with id 6b9fc909dfd5fdbb76efe43b147a82154e549d89122362348a1e11a7bd0bdbd7 Jun 08 16:49:28.148341 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:28.148301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" event={"ID":"2489ac05-ea97-4034-aefa-4f87450babbf","Type":"ContainerStarted","Data":"6b9fc909dfd5fdbb76efe43b147a82154e549d89122362348a1e11a7bd0bdbd7"} Jun 08 16:49:29.154448 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:29.154412 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" event={"ID":"2489ac05-ea97-4034-aefa-4f87450babbf","Type":"ContainerStarted","Data":"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e"} Jun 08 16:49:29.170476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:49:29.170420 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" podStartSLOduration=1.7954623669999998 podStartE2EDuration="2.170403321s" totalImagesPullingTime="374.940954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:49:29.170034678 +0000 UTC m=+973.556223069" watchObservedRunningTime="2026-06-08 16:49:29.170403321 +0000 UTC m=+973.556591714" Jun 08 16:51:56.012168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.012134 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-796674c885-7bxq5"] Jun 08 16:51:56.015767 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.015738 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.036390 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.036348 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-796674c885-7bxq5"] Jun 08 16:51:56.116692 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.116627 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/e265f4f6-8378-48a6-8766-206f3d1e346a-oidc-ca\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.116981 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.116791 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e265f4f6-8378-48a6-8766-206f3d1e346a-tls-cert\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.116981 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.116835 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdck\" (UniqueName: \"kubernetes.io/projected/e265f4f6-8378-48a6-8766-206f3d1e346a-kube-api-access-cwdck\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.218140 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.218089 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/e265f4f6-8378-48a6-8766-206f3d1e346a-oidc-ca\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.218349 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.218205 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e265f4f6-8378-48a6-8766-206f3d1e346a-tls-cert\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.218349 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.218235 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdck\" (UniqueName: \"kubernetes.io/projected/e265f4f6-8378-48a6-8766-206f3d1e346a-kube-api-access-cwdck\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.218834 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.218812 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/e265f4f6-8378-48a6-8766-206f3d1e346a-oidc-ca\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.220915 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.220893 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e265f4f6-8378-48a6-8766-206f3d1e346a-tls-cert\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.228207 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.228176 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdck\" (UniqueName: \"kubernetes.io/projected/e265f4f6-8378-48a6-8766-206f3d1e346a-kube-api-access-cwdck\") pod \"authorino-796674c885-7bxq5\" (UID: \"e265f4f6-8378-48a6-8766-206f3d1e346a\") " pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.325276 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.325178 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-796674c885-7bxq5" Jun 08 16:51:56.491976 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.491946 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-796674c885-7bxq5"] Jun 08 16:51:56.493381 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:51:56.493350 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode265f4f6_8378_48a6_8766_206f3d1e346a.slice/crio-bc672b662ed3d2d8d6cce5bf8cf605cf47dfb4cfd80cbcd8c043c56aa64897a7 WatchSource:0}: Error finding container bc672b662ed3d2d8d6cce5bf8cf605cf47dfb4cfd80cbcd8c043c56aa64897a7: Status 404 returned error can't find the container with id bc672b662ed3d2d8d6cce5bf8cf605cf47dfb4cfd80cbcd8c043c56aa64897a7 Jun 08 16:51:56.495008 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.494985 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 16:51:56.870492 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:56.870453 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796674c885-7bxq5" event={"ID":"e265f4f6-8378-48a6-8766-206f3d1e346a","Type":"ContainerStarted","Data":"bc672b662ed3d2d8d6cce5bf8cf605cf47dfb4cfd80cbcd8c043c56aa64897a7"} Jun 08 16:51:57.878503 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:57.878463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-796674c885-7bxq5" event={"ID":"e265f4f6-8378-48a6-8766-206f3d1e346a","Type":"ContainerStarted","Data":"e0f53ff3e7204e969343ae56dd9263b06f7b3951737ac601175f42ccf565f4ef"} Jun 08 16:51:57.928471 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:57.928415 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-796674c885-7bxq5" podStartSLOduration=2.490107552 podStartE2EDuration="2.928396871s" totalImagesPullingTime="438.289319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:51:57.923824529 +0000 UTC m=+1122.310012914" watchObservedRunningTime="2026-06-08 16:51:57.928396871 +0000 UTC m=+1122.314585265" Jun 08 16:51:58.013767 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.013733 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:51:58.013923 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.013901 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" podUID="2489ac05-ea97-4034-aefa-4f87450babbf" containerName="authorino" containerID="cri-o://06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e" gracePeriod=30 Jun 08 16:51:58.273016 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.272987 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:51:58.341348 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.341305 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert\") pod \"2489ac05-ea97-4034-aefa-4f87450babbf\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " Jun 08 16:51:58.341534 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.341396 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca\") pod \"2489ac05-ea97-4034-aefa-4f87450babbf\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " Jun 08 16:51:58.341534 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.341480 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2x8n\" (UniqueName: \"kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n\") pod \"2489ac05-ea97-4034-aefa-4f87450babbf\" (UID: \"2489ac05-ea97-4034-aefa-4f87450babbf\") " Jun 08 16:51:58.343674 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.343604 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n" (OuterVolumeSpecName: "kube-api-access-h2x8n") pod "2489ac05-ea97-4034-aefa-4f87450babbf" (UID: "2489ac05-ea97-4034-aefa-4f87450babbf"). InnerVolumeSpecName "kube-api-access-h2x8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:51:58.346851 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.346823 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "2489ac05-ea97-4034-aefa-4f87450babbf" (UID: "2489ac05-ea97-4034-aefa-4f87450babbf"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 16:51:58.353970 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.353937 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2489ac05-ea97-4034-aefa-4f87450babbf" (UID: "2489ac05-ea97-4034-aefa-4f87450babbf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:51:58.442833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.442739 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2x8n\" (UniqueName: \"kubernetes.io/projected/2489ac05-ea97-4034-aefa-4f87450babbf-kube-api-access-h2x8n\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:51:58.442833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.442773 2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2489ac05-ea97-4034-aefa-4f87450babbf-tls-cert\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:51:58.442833 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.442783 2559 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/2489ac05-ea97-4034-aefa-4f87450babbf-oidc-ca\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:51:58.885191 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.885153 2559 generic.go:356] "Generic (PLEG): container finished" podID="2489ac05-ea97-4034-aefa-4f87450babbf" containerID="06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e" exitCode=0 Jun 08 16:51:58.885191 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.885195 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" Jun 08 16:51:58.885739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.885237 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" event={"ID":"2489ac05-ea97-4034-aefa-4f87450babbf","Type":"ContainerDied","Data":"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e"} Jun 08 16:51:58.885739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.885281 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fd874b6b8-zh4pc" event={"ID":"2489ac05-ea97-4034-aefa-4f87450babbf","Type":"ContainerDied","Data":"6b9fc909dfd5fdbb76efe43b147a82154e549d89122362348a1e11a7bd0bdbd7"} Jun 08 16:51:58.885739 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.885302 2559 scope.go:117] "RemoveContainer" containerID="06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e" Jun 08 16:51:58.895289 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.895263 2559 scope.go:117] "RemoveContainer" containerID="06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e" Jun 08 16:51:58.895685 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:51:58.895627 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e\": container with ID starting with 06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e not found: ID does not exist" containerID="06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e" Jun 08 16:51:58.895772 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.895697 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e"} err="failed to get container status \"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e\": rpc error: code = NotFound desc = could not find container \"06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e\": container with ID starting with 06e7f9b9c568296b39a9c1343d95be969b6e231c1ea7bb57fedb7a191346829e not found: ID does not exist" Jun 08 16:51:58.919311 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.919275 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:51:58.939609 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:51:58.939577 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-fd874b6b8-zh4pc"] Jun 08 16:52:00.118361 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:52:00.118329 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2489ac05-ea97-4034-aefa-4f87450babbf" path="/var/lib/kubelet/pods/2489ac05-ea97-4034-aefa-4f87450babbf/volumes" Jun 08 16:53:16.160129 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:16.160097 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:53:16.160750 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:16.160726 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:53:27.347987 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.347950 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:53:27.348382 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.348201 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" podUID="490403e1-688e-4cea-8233-14bcc0cf6d02" containerName="manager" containerID="cri-o://aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff" gracePeriod=30 Jun 08 16:53:27.599808 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.599739 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:53:27.684476 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.684439 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") pod \"490403e1-688e-4cea-8233-14bcc0cf6d02\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " Jun 08 16:53:27.684681 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.684602 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxsjw\" (UniqueName: \"kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw\") pod \"490403e1-688e-4cea-8233-14bcc0cf6d02\" (UID: \"490403e1-688e-4cea-8233-14bcc0cf6d02\") " Jun 08 16:53:27.686917 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.686884 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "490403e1-688e-4cea-8233-14bcc0cf6d02" (UID: "490403e1-688e-4cea-8233-14bcc0cf6d02"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 16:53:27.687030 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.686978 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw" (OuterVolumeSpecName: "kube-api-access-jxsjw") pod "490403e1-688e-4cea-8233-14bcc0cf6d02" (UID: "490403e1-688e-4cea-8233-14bcc0cf6d02"). InnerVolumeSpecName "kube-api-access-jxsjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 16:53:27.785852 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.785794 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxsjw\" (UniqueName: \"kubernetes.io/projected/490403e1-688e-4cea-8233-14bcc0cf6d02-kube-api-access-jxsjw\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:53:27.785852 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:27.785846 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/490403e1-688e-4cea-8233-14bcc0cf6d02-webhook-cert\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 16:53:28.323168 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.323135 2559 generic.go:356] "Generic (PLEG): container finished" podID="490403e1-688e-4cea-8233-14bcc0cf6d02" containerID="aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff" exitCode=0 Jun 08 16:53:28.323371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.323188 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" Jun 08 16:53:28.323371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.323229 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" event={"ID":"490403e1-688e-4cea-8233-14bcc0cf6d02","Type":"ContainerDied","Data":"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff"} Jun 08 16:53:28.323371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.323294 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-7hmbq" event={"ID":"490403e1-688e-4cea-8233-14bcc0cf6d02","Type":"ContainerDied","Data":"47b74607be11fb8a84e80aa0c5586232620f0d825acfe019e42e8f377489b186"} Jun 08 16:53:28.323371 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.323326 2559 scope.go:117] "RemoveContainer" containerID="aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff" Jun 08 16:53:28.332411 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.332391 2559 scope.go:117] "RemoveContainer" containerID="aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff" Jun 08 16:53:28.332760 ip-10-0-141-92 kubenswrapper[2559]: E0608 16:53:28.332739 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff\": container with ID starting with aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff not found: ID does not exist" containerID="aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff" Jun 08 16:53:28.332842 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.332770 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff"} err="failed to get container status \"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff\": rpc error: code = NotFound desc = could not find container \"aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff\": container with ID starting with aa4e728dc7fbc4e4203004657670d7a18c34be69e6684e73a4e16024c10c78ff not found: ID does not exist" Jun 08 16:53:28.343101 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.343067 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:53:28.349753 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:28.349724 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-7hmbq"] Jun 08 16:53:29.115345 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115307 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76f9db8d84-tnkdr"] Jun 08 16:53:29.115741 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115727 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2489ac05-ea97-4034-aefa-4f87450babbf" containerName="authorino" Jun 08 16:53:29.115792 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115743 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2489ac05-ea97-4034-aefa-4f87450babbf" containerName="authorino" Jun 08 16:53:29.115792 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115758 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="490403e1-688e-4cea-8233-14bcc0cf6d02" containerName="manager" Jun 08 16:53:29.115792 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115764 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="490403e1-688e-4cea-8233-14bcc0cf6d02" containerName="manager" Jun 08 16:53:29.115882 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115830 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="490403e1-688e-4cea-8233-14bcc0cf6d02" containerName="manager" Jun 08 16:53:29.115882 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.115841 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2489ac05-ea97-4034-aefa-4f87450babbf" containerName="authorino" Jun 08 16:53:29.120255 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.120234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.123881 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.123856 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 08 16:53:29.125124 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.125103 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qgwwh\"" Jun 08 16:53:29.125235 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.125145 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 08 16:53:29.131687 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.131663 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-tnkdr"] Jun 08 16:53:29.200198 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.200162 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e263d322-62d9-4972-9f92-13cc503823de-webhook-cert\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.200382 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.200248 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxv5d\" (UniqueName: \"kubernetes.io/projected/e263d322-62d9-4972-9f92-13cc503823de-kube-api-access-dxv5d\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.301121 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.301081 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxv5d\" (UniqueName: \"kubernetes.io/projected/e263d322-62d9-4972-9f92-13cc503823de-kube-api-access-dxv5d\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.301355 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.301173 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e263d322-62d9-4972-9f92-13cc503823de-webhook-cert\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.303893 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.303868 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e263d322-62d9-4972-9f92-13cc503823de-webhook-cert\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.311501 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.311479 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxv5d\" (UniqueName: \"kubernetes.io/projected/e263d322-62d9-4972-9f92-13cc503823de-kube-api-access-dxv5d\") pod \"maas-controller-76f9db8d84-tnkdr\" (UID: \"e263d322-62d9-4972-9f92-13cc503823de\") " pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.430580 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.430475 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:29.574283 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:29.574254 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76f9db8d84-tnkdr"] Jun 08 16:53:29.576680 ip-10-0-141-92 kubenswrapper[2559]: W0608 16:53:29.576601 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode263d322_62d9_4972_9f92_13cc503823de.slice/crio-e61d4dd9bc715c18a5f4f2f62b3cf71d3d78a43db22a2f9edb741cafd7bfed59 WatchSource:0}: Error finding container e61d4dd9bc715c18a5f4f2f62b3cf71d3d78a43db22a2f9edb741cafd7bfed59: Status 404 returned error can't find the container with id e61d4dd9bc715c18a5f4f2f62b3cf71d3d78a43db22a2f9edb741cafd7bfed59 Jun 08 16:53:30.119761 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:30.119728 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="490403e1-688e-4cea-8233-14bcc0cf6d02" path="/var/lib/kubelet/pods/490403e1-688e-4cea-8233-14bcc0cf6d02/volumes" Jun 08 16:53:30.336134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:30.336042 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" event={"ID":"e263d322-62d9-4972-9f92-13cc503823de","Type":"ContainerStarted","Data":"a499750ff230b17fd5cdca1dc20eed023dfc776350104a03b7223074dca4f1ce"} Jun 08 16:53:30.336134 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:30.336093 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" event={"ID":"e263d322-62d9-4972-9f92-13cc503823de","Type":"ContainerStarted","Data":"e61d4dd9bc715c18a5f4f2f62b3cf71d3d78a43db22a2f9edb741cafd7bfed59"} Jun 08 16:53:30.336329 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:30.336156 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:53:30.355402 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:30.355349 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" podStartSLOduration=0.907296707 podStartE2EDuration="1.355331727s" totalImagesPullingTime="448.03502ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 16:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 16:53:30.353443174 +0000 UTC m=+1214.739631604" watchObservedRunningTime="2026-06-08 16:53:30.355331727 +0000 UTC m=+1214.741520121" Jun 08 16:53:41.346947 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:53:41.346917 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76f9db8d84-tnkdr" Jun 08 16:58:16.190155 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:58:16.190114 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 16:58:16.192496 ip-10-0-141-92 kubenswrapper[2559]: I0608 16:58:16.192473 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:00:00.143455 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.143421 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29682300-97j2d"] Jun 08 17:00:00.146998 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.146981 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:00.149696 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.149675 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vk2xp\"" Jun 08 17:00:00.162245 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.162215 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29682300-97j2d"] Jun 08 17:00:00.216845 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.216802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4fdk\" (UniqueName: \"kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk\") pod \"maas-api-key-cleanup-29682300-97j2d\" (UID: \"2c5df3f0-90ea-4436-8158-2b553bb62918\") " pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:00.317718 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.317675 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4fdk\" (UniqueName: \"kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk\") pod \"maas-api-key-cleanup-29682300-97j2d\" (UID: \"2c5df3f0-90ea-4436-8158-2b553bb62918\") " pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:00.326915 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.326880 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4fdk\" (UniqueName: \"kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk\") pod \"maas-api-key-cleanup-29682300-97j2d\" (UID: \"2c5df3f0-90ea-4436-8158-2b553bb62918\") " pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:00.456919 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.456828 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:00.594233 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.594204 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29682300-97j2d"] Jun 08 17:00:00.595938 ip-10-0-141-92 kubenswrapper[2559]: W0608 17:00:00.595912 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5df3f0_90ea_4436_8158_2b553bb62918.slice/crio-89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44 WatchSource:0}: Error finding container 89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44: Status 404 returned error can't find the container with id 89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44 Jun 08 17:00:00.597930 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:00.597913 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 17:00:01.231230 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:01.231198 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" event={"ID":"2c5df3f0-90ea-4436-8158-2b553bb62918","Type":"ContainerStarted","Data":"89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44"} Jun 08 17:00:05.254345 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:05.254307 2559 generic.go:356] "Generic (PLEG): container finished" podID="2c5df3f0-90ea-4436-8158-2b553bb62918" containerID="77c1a13b508476c626f458bcdbb2d4c2996cab87e54a2891edff97a788bd1806" exitCode=0 Jun 08 17:00:05.254778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:05.254394 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" event={"ID":"2c5df3f0-90ea-4436-8158-2b553bb62918","Type":"ContainerDied","Data":"77c1a13b508476c626f458bcdbb2d4c2996cab87e54a2891edff97a788bd1806"} Jun 08 17:00:06.395296 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:06.395266 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:06.580009 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:06.579920 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4fdk\" (UniqueName: \"kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk\") pod \"2c5df3f0-90ea-4436-8158-2b553bb62918\" (UID: \"2c5df3f0-90ea-4436-8158-2b553bb62918\") " Jun 08 17:00:06.582145 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:06.582104 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk" (OuterVolumeSpecName: "kube-api-access-h4fdk") pod "2c5df3f0-90ea-4436-8158-2b553bb62918" (UID: "2c5df3f0-90ea-4436-8158-2b553bb62918"). InnerVolumeSpecName "kube-api-access-h4fdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:00:06.681154 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:06.681108 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4fdk\" (UniqueName: \"kubernetes.io/projected/2c5df3f0-90ea-4436-8158-2b553bb62918-kube-api-access-h4fdk\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:00:07.265624 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:07.265592 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" Jun 08 17:00:07.265842 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:07.265614 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682300-97j2d" event={"ID":"2c5df3f0-90ea-4436-8158-2b553bb62918","Type":"ContainerDied","Data":"89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44"} Jun 08 17:00:07.265842 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:00:07.265693 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89adf08bd1c80a10fa38fb34361cf513572bd9aa67789b3eed4f5e82a4cbbd44" Jun 08 17:03:16.220682 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:03:16.220652 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:03:16.224559 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:03:16.224536 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:08:16.255108 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:08:16.255080 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:08:16.259580 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:08:16.259562 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:13:16.284930 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:13:16.284898 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:13:16.292961 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:13:16.292939 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tlf2q_5ed14746-301c-4904-875c-8ba3415aaf7c/ovn-acl-logging/0.log" Jun 08 17:14:32.576060 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.576023 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:32.576573 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.576445 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c5df3f0-90ea-4436-8158-2b553bb62918" containerName="cleanup" Jun 08 17:14:32.576573 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.576456 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c5df3f0-90ea-4436-8158-2b553bb62918" containerName="cleanup" Jun 08 17:14:32.576573 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.576544 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c5df3f0-90ea-4436-8158-2b553bb62918" containerName="cleanup" Jun 08 17:14:32.579987 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.579963 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.582756 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.582731 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-83a91df9-openshift-default-dockercfg-r8b79\"" Jun 08 17:14:32.600978 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.600949 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:32.613964 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.613929 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614213 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614033 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614213 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614073 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614213 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614102 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614213 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614128 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614213 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bzwm\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614323 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.614513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.614380 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715553 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715511 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715557 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715588 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715719 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bzwm\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.715778 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716043 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716043 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716043 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.715927 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716043 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.716024 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716237 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.716063 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716237 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.716120 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716337 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.716237 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.716660 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.716599 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.718549 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.718525 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.719034 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.719006 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.730914 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.730855 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.733662 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.733608 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bzwm\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm\") pod \"e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:32.891225 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:32.891186 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:33.035764 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.035735 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:33.037727 ip-10-0-141-92 kubenswrapper[2559]: W0608 17:14:33.037698 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69457b91_af86_4746_8186_5b56f5b4e537.slice/crio-7484928253c5985fec739b6a9bf1546945b9dd1795f335c4267511a9fc7d4460 WatchSource:0}: Error finding container 7484928253c5985fec739b6a9bf1546945b9dd1795f335c4267511a9fc7d4460: Status 404 returned error can't find the container with id 7484928253c5985fec739b6a9bf1546945b9dd1795f335c4267511a9fc7d4460 Jun 08 17:14:33.039812 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.039776 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:33.039911 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.039850 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:33.039911 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.039878 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:33.507416 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.507377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" event={"ID":"69457b91-af86-4746-8186-5b56f5b4e537","Type":"ContainerStarted","Data":"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37"} Jun 08 17:14:33.507623 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.507428 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" event={"ID":"69457b91-af86-4746-8186-5b56f5b4e537","Type":"ContainerStarted","Data":"7484928253c5985fec739b6a9bf1546945b9dd1795f335c4267511a9fc7d4460"} Jun 08 17:14:33.530453 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.530397 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" podStartSLOduration=1.530382543 podStartE2EDuration="1.530382543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 17:14:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 17:14:33.527865772 +0000 UTC m=+2477.914054185" watchObservedRunningTime="2026-06-08 17:14:33.530382543 +0000 UTC m=+2477.916570937" Jun 08 17:14:33.891540 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:33.891489 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:34.897571 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:34.897537 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:35.516873 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:35.516834 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:35.517945 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:35.517926 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:39.011008 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:39.010970 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:39.011536 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:39.011159 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" podUID="69457b91-af86-4746-8186-5b56f5b4e537" containerName="istio-proxy" containerID="cri-o://335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37" gracePeriod=30 Jun 08 17:14:44.274742 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.274661 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:44.318794 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318759 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318812 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318850 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318888 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318908 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318937 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318965 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319006 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.318990 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319014 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bzwm\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm\") pod \"69457b91-af86-4746-8186-5b56f5b4e537\" (UID: \"69457b91-af86-4746-8186-5b56f5b4e537\") " Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319086 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319234 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319318 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319323 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-socket\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.319386 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319256 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 17:14:44.319680 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319391 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-workload-certs\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.319680 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.319405 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data" (OuterVolumeSpecName: "istio-data") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:44.321538 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.321511 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm" (OuterVolumeSpecName: "kube-api-access-2bzwm") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "kube-api-access-2bzwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:14:44.321965 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.321941 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token" (OuterVolumeSpecName: "istio-token") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:14:44.321965 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.321943 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 08 17:14:44.322097 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.321989 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69457b91-af86-4746-8186-5b56f5b4e537" (UID: "69457b91-af86-4746-8186-5b56f5b4e537"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:44.420381 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420335 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-istio-token\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420381 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420373 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-data\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420381 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420385 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-istio-envoy\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420627 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420395 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69457b91-af86-4746-8186-5b56f5b4e537-istio-podinfo\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420627 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420406 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69457b91-af86-4746-8186-5b56f5b4e537-credential-socket\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420627 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420419 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bzwm\" (UniqueName: \"kubernetes.io/projected/69457b91-af86-4746-8186-5b56f5b4e537-kube-api-access-2bzwm\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.420627 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.420431 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69457b91-af86-4746-8186-5b56f5b4e537-istiod-ca-cert\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:44.561728 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.561615 2559 generic.go:356] "Generic (PLEG): container finished" podID="69457b91-af86-4746-8186-5b56f5b4e537" containerID="335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37" exitCode=0 Jun 08 17:14:44.561728 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.561683 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" Jun 08 17:14:44.561728 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.561707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" event={"ID":"69457b91-af86-4746-8186-5b56f5b4e537","Type":"ContainerDied","Data":"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37"} Jun 08 17:14:44.561995 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.561765 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b" event={"ID":"69457b91-af86-4746-8186-5b56f5b4e537","Type":"ContainerDied","Data":"7484928253c5985fec739b6a9bf1546945b9dd1795f335c4267511a9fc7d4460"} Jun 08 17:14:44.561995 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.561793 2559 scope.go:117] "RemoveContainer" containerID="335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37" Jun 08 17:14:44.571385 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.571366 2559 scope.go:117] "RemoveContainer" containerID="335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37" Jun 08 17:14:44.571680 ip-10-0-141-92 kubenswrapper[2559]: E0608 17:14:44.571656 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37\": container with ID starting with 335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37 not found: ID does not exist" containerID="335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37" Jun 08 17:14:44.571738 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.571690 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37"} err="failed to get container status \"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37\": rpc error: code = NotFound desc = could not find container \"335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37\": container with ID starting with 335ad4ebb6b7271da8417ab8dd6e061b13edd2472fcddaeaf09fdcc696271b37 not found: ID does not exist" Jun 08 17:14:44.584587 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.584560 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:44.589823 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:44.589784 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-83a91df9-openshift-default-f9fd8fcd4-9l47b"] Jun 08 17:14:45.141266 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.141233 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:14:45.141685 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.141668 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69457b91-af86-4746-8186-5b56f5b4e537" containerName="istio-proxy" Jun 08 17:14:45.141743 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.141687 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="69457b91-af86-4746-8186-5b56f5b4e537" containerName="istio-proxy" Jun 08 17:14:45.141795 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.141785 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="69457b91-af86-4746-8186-5b56f5b4e537" containerName="istio-proxy" Jun 08 17:14:45.146341 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.146315 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.148960 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.148933 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0a5a71e1-openshift-default-dockercfg-ldpwc\"" Jun 08 17:14:45.158414 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.158388 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:14:45.228023 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.227978 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228218 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228072 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228218 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228090 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228218 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228113 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228218 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228129 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228218 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228144 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcgw\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228273 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228310 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.228513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.228378 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.329668 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329595 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329849 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329885 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329920 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329944 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.329970 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcgw\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330148 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330063 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330572 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330300 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330723 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330596 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330723 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330673 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.330723 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.330690 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.331194 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.331170 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.332512 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.332491 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.332830 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.332814 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.338435 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.338404 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.338528 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.338486 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcgw\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw\") pod \"e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.457694 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.457568 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:45.606570 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.606539 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:14:45.608100 ip-10-0-141-92 kubenswrapper[2559]: W0608 17:14:45.608070 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d52b8ad_e597_4743_a7cc_7d9493fe9cbb.slice/crio-e5e0be6177379fb3a2d6480564a5b8d0bdaf91866a61323f960db4c59e4cb80a WatchSource:0}: Error finding container e5e0be6177379fb3a2d6480564a5b8d0bdaf91866a61323f960db4c59e4cb80a: Status 404 returned error can't find the container with id e5e0be6177379fb3a2d6480564a5b8d0bdaf91866a61323f960db4c59e4cb80a Jun 08 17:14:45.610493 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.610458 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:45.610548 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.610534 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:45.610588 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:45.610571 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jun 08 17:14:46.120031 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:46.120000 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69457b91-af86-4746-8186-5b56f5b4e537" path="/var/lib/kubelet/pods/69457b91-af86-4746-8186-5b56f5b4e537/volumes" Jun 08 17:14:46.576655 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:46.576545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" event={"ID":"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb","Type":"ContainerStarted","Data":"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0"} Jun 08 17:14:46.576655 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:46.576603 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" event={"ID":"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb","Type":"ContainerStarted","Data":"e5e0be6177379fb3a2d6480564a5b8d0bdaf91866a61323f960db4c59e4cb80a"} Jun 08 17:14:46.598922 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:46.598870 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" podStartSLOduration=1.598851305 podStartE2EDuration="1.598851305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 17:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 17:14:46.596807343 +0000 UTC m=+2490.982995738" watchObservedRunningTime="2026-06-08 17:14:46.598851305 +0000 UTC m=+2490.985039699" Jun 08 17:14:47.458761 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:47.458718 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:47.464946 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:47.464917 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:47.588829 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:47.588788 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:47.590076 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:47.590053 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:52.570122 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:52.570080 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:14:52.570677 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:52.570261 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" podUID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" containerName="istio-proxy" containerID="cri-o://0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0" gracePeriod=30 Jun 08 17:14:57.831611 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.831541 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:57.950238 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950196 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950441 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950268 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950441 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950308 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trcgw\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950441 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950367 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950617 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950510 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950617 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950525 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:57.950617 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950552 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950617 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950600 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950861 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950675 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950861 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950719 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data\") pod \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\" (UID: \"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb\") " Jun 08 17:14:57.950861 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.950797 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 17:14:57.951163 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.951133 2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-certs\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:57.951256 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.951158 2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istiod-ca-cert\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:57.951256 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.951173 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:57.951343 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.951315 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:57.951379 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.951354 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:57.952751 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.952729 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 08 17:14:57.952751 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.952742 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw" (OuterVolumeSpecName: "kube-api-access-trcgw") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "kube-api-access-trcgw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:14:57.953253 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.953227 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:14:57.953253 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:57.953240 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" (UID: "2d52b8ad-e597-4743-a7cc-7d9493fe9cbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 17:14:58.052516 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052482 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trcgw\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-kube-api-access-trcgw\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052516 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052511 2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-envoy\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052516 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052521 2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-token\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052788 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052529 2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-workload-socket\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052788 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052537 2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-credential-socket\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052788 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052546 2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-data\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.052788 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.052556 2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb-istio-podinfo\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:14:58.645807 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.645768 2559 generic.go:356] "Generic (PLEG): container finished" podID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" containerID="0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0" exitCode=0 Jun 08 17:14:58.645982 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.645822 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" Jun 08 17:14:58.645982 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.645836 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" event={"ID":"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb","Type":"ContainerDied","Data":"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0"} Jun 08 17:14:58.645982 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.645891 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z" event={"ID":"2d52b8ad-e597-4743-a7cc-7d9493fe9cbb","Type":"ContainerDied","Data":"e5e0be6177379fb3a2d6480564a5b8d0bdaf91866a61323f960db4c59e4cb80a"} Jun 08 17:14:58.645982 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.645921 2559 scope.go:117] "RemoveContainer" containerID="0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0" Jun 08 17:14:58.655288 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.655264 2559 scope.go:117] "RemoveContainer" containerID="0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0" Jun 08 17:14:58.655573 ip-10-0-141-92 kubenswrapper[2559]: E0608 17:14:58.655555 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0\": container with ID starting with 0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0 not found: ID does not exist" containerID="0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0" Jun 08 17:14:58.655624 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.655584 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0"} err="failed to get container status \"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0\": rpc error: code = NotFound desc = could not find container \"0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0\": container with ID starting with 0929f8aabcdabf08782c600c3cc3d50b460befbdb13e06b4825398b977de8bb0 not found: ID does not exist" Jun 08 17:14:58.665289 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.665259 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:14:58.669317 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:14:58.669287 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0a5a71e1-openshift-default-65db9fc588-8l57z"] Jun 08 17:15:00.118772 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.118737 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" path="/var/lib/kubelet/pods/2d52b8ad-e597-4743-a7cc-7d9493fe9cbb/volumes" Jun 08 17:15:00.133822 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.133787 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29682315-7b2b9"] Jun 08 17:15:00.134224 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.134211 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" containerName="istio-proxy" Jun 08 17:15:00.134268 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.134227 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" containerName="istio-proxy" Jun 08 17:15:00.134315 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.134305 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d52b8ad-e597-4743-a7cc-7d9493fe9cbb" containerName="istio-proxy" Jun 08 17:15:00.138843 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.138824 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:00.141858 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.141834 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vk2xp\"" Jun 08 17:15:00.144202 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.144177 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29682315-7b2b9"] Jun 08 17:15:00.273356 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.273318 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bk49\" (UniqueName: \"kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49\") pod \"maas-api-key-cleanup-29682315-7b2b9\" (UID: \"8e635853-fdb8-4a11-9080-c4a3fc50e27c\") " pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:00.374269 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.374176 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bk49\" (UniqueName: \"kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49\") pod \"maas-api-key-cleanup-29682315-7b2b9\" (UID: \"8e635853-fdb8-4a11-9080-c4a3fc50e27c\") " pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:00.382807 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.382774 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bk49\" (UniqueName: \"kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49\") pod \"maas-api-key-cleanup-29682315-7b2b9\" (UID: \"8e635853-fdb8-4a11-9080-c4a3fc50e27c\") " pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:00.451325 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.451290 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:00.589171 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.589145 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29682315-7b2b9"] Jun 08 17:15:00.591317 ip-10-0-141-92 kubenswrapper[2559]: W0608 17:15:00.591292 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e635853_fdb8_4a11_9080_c4a3fc50e27c.slice/crio-4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243 WatchSource:0}: Error finding container 4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243: Status 404 returned error can't find the container with id 4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243 Jun 08 17:15:00.657664 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:00.657563 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" event={"ID":"8e635853-fdb8-4a11-9080-c4a3fc50e27c","Type":"ContainerStarted","Data":"4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243"} Jun 08 17:15:01.664522 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:01.664485 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" event={"ID":"8e635853-fdb8-4a11-9080-c4a3fc50e27c","Type":"ContainerStarted","Data":"3bcaf4317d8f91b8c816dcefc54724ec7137f01a8584bf926d022aebb33d0684"} Jun 08 17:15:02.670943 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:02.670843 2559 generic.go:356] "Generic (PLEG): container finished" podID="8e635853-fdb8-4a11-9080-c4a3fc50e27c" containerID="3bcaf4317d8f91b8c816dcefc54724ec7137f01a8584bf926d022aebb33d0684" exitCode=0 Jun 08 17:15:02.671385 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:02.670939 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" event={"ID":"8e635853-fdb8-4a11-9080-c4a3fc50e27c","Type":"ContainerDied","Data":"3bcaf4317d8f91b8c816dcefc54724ec7137f01a8584bf926d022aebb33d0684"} Jun 08 17:15:03.815235 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:03.815209 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:03.907703 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:03.907623 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bk49\" (UniqueName: \"kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49\") pod \"8e635853-fdb8-4a11-9080-c4a3fc50e27c\" (UID: \"8e635853-fdb8-4a11-9080-c4a3fc50e27c\") " Jun 08 17:15:03.909979 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:03.909948 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49" (OuterVolumeSpecName: "kube-api-access-2bk49") pod "8e635853-fdb8-4a11-9080-c4a3fc50e27c" (UID: "8e635853-fdb8-4a11-9080-c4a3fc50e27c"). InnerVolumeSpecName "kube-api-access-2bk49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 17:15:04.008674 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:04.008550 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bk49\" (UniqueName: \"kubernetes.io/projected/8e635853-fdb8-4a11-9080-c4a3fc50e27c-kube-api-access-2bk49\") on node \"ip-10-0-141-92.ec2.internal\" DevicePath \"\"" Jun 08 17:15:04.681750 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:04.681727 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" Jun 08 17:15:04.681911 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:04.681716 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29682315-7b2b9" event={"ID":"8e635853-fdb8-4a11-9080-c4a3fc50e27c","Type":"ContainerDied","Data":"4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243"} Jun 08 17:15:04.681911 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:04.681877 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a6d50c1c040778478643955a83fc0cc6eb0cf82aed8418ce010cca68691c243" Jun 08 17:15:06.118960 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:06.118928 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-796674c885-7bxq5_e265f4f6-8378-48a6-8766-206f3d1e346a/authorino/0.log" Jun 08 17:15:10.685738 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:10.685707 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-75b7f6c686-8s9lc_5261595f-b064-4015-bada-3748f3fb6fd5/nginx/0.log" Jun 08 17:15:11.155222 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.155185 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29682300-97j2d_2c5df3f0-90ea-4436-8158-2b553bb62918/cleanup/0.log" Jun 08 17:15:11.266922 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.266888 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29682315-7b2b9_8e635853-fdb8-4a11-9080-c4a3fc50e27c/cleanup/0.log" Jun 08 17:15:11.388087 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.388055 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f9db8d84-tnkdr_e263d322-62d9-4972-9f92-13cc503823de/manager/0.log" Jun 08 17:15:11.495680 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.495573 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-2hkjc_2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139/server/0.log" Jun 08 17:15:11.863478 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.863452 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf5fd9654-ht8h8_dfbf5980-9001-430b-abe8-c57cb88c4dc5/copy-manifests/0.log" Jun 08 17:15:11.871224 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:11.871191 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf5fd9654-ht8h8_dfbf5980-9001-430b-abe8-c57cb88c4dc5/manager/0.log" Jun 08 17:15:12.099399 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:12.099367 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-424jq_67fd4edf-95bd-4180-961d-8671251665e8/postgres/0.log" Jun 08 17:15:13.384051 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:13.384009 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-796674c885-7bxq5_e265f4f6-8378-48a6-8766-206f3d1e346a/authorino/0.log" Jun 08 17:15:13.516336 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:13.516301 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rh996_285835d0-dbd4-4fff-a6fe-a7d1f954e297/manager/0.log" Jun 08 17:15:13.859761 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:13.859723 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-m56xq_7766c1c7-0747-4391-af89-7ea28f9d9fb6/registry-server/0.log" Jun 08 17:15:14.561732 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:14.561702 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795jnn2b_90bf27fa-47f1-4f79-9e73-44f54baf2ec4/istio-proxy/0.log" Jun 08 17:15:15.035881 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:15.035845 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/1.log" Jun 08 17:15:15.157935 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:15.157903 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-x72c7_ca2483f0-0110-40c0-b699-13404f7be2e0/payload-pre-processing/0.log" Jun 08 17:15:15.393568 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:15.393540 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 17:15:20.166759 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.166726 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l74s4/must-gather-pzkjr"] Jun 08 17:15:20.167219 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.167199 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e635853-fdb8-4a11-9080-c4a3fc50e27c" containerName="cleanup" Jun 08 17:15:20.167293 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.167222 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e635853-fdb8-4a11-9080-c4a3fc50e27c" containerName="cleanup" Jun 08 17:15:20.167352 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.167330 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e635853-fdb8-4a11-9080-c4a3fc50e27c" containerName="cleanup" Jun 08 17:15:20.170729 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.170702 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.173510 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.173489 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l74s4\"/\"openshift-service-ca.crt\"" Jun 08 17:15:20.173621 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.173540 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l74s4\"/\"kube-root-ca.crt\"" Jun 08 17:15:20.174802 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.174783 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l74s4\"/\"default-dockercfg-c4z6q\"" Jun 08 17:15:20.178726 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.178702 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l74s4/must-gather-pzkjr"] Jun 08 17:15:20.264671 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.264603 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28b6f895-fe2f-47b7-926b-4675c409027c-must-gather-output\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.264883 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.264718 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsw5l\" (UniqueName: \"kubernetes.io/projected/28b6f895-fe2f-47b7-926b-4675c409027c-kube-api-access-hsw5l\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.365514 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.365476 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28b6f895-fe2f-47b7-926b-4675c409027c-must-gather-output\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.365709 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.365527 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsw5l\" (UniqueName: \"kubernetes.io/projected/28b6f895-fe2f-47b7-926b-4675c409027c-kube-api-access-hsw5l\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.365895 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.365874 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/28b6f895-fe2f-47b7-926b-4675c409027c-must-gather-output\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.380649 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.373744 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsw5l\" (UniqueName: \"kubernetes.io/projected/28b6f895-fe2f-47b7-926b-4675c409027c-kube-api-access-hsw5l\") pod \"must-gather-pzkjr\" (UID: \"28b6f895-fe2f-47b7-926b-4675c409027c\") " pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.482014 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.481922 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l74s4/must-gather-pzkjr" Jun 08 17:15:20.619564 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.619536 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l74s4/must-gather-pzkjr"] Jun 08 17:15:20.621551 ip-10-0-141-92 kubenswrapper[2559]: W0608 17:15:20.621516 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28b6f895_fe2f_47b7_926b_4675c409027c.slice/crio-7fc8551933da6890793e4c0d49bb69d0bc43361f2004b028d8a36c5265204c00 WatchSource:0}: Error finding container 7fc8551933da6890793e4c0d49bb69d0bc43361f2004b028d8a36c5265204c00: Status 404 returned error can't find the container with id 7fc8551933da6890793e4c0d49bb69d0bc43361f2004b028d8a36c5265204c00 Jun 08 17:15:20.623409 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.623390 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 17:15:20.761436 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:20.761343 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l74s4/must-gather-pzkjr" event={"ID":"28b6f895-fe2f-47b7-926b-4675c409027c","Type":"ContainerStarted","Data":"7fc8551933da6890793e4c0d49bb69d0bc43361f2004b028d8a36c5265204c00"} Jun 08 17:15:21.771217 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:21.771072 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l74s4/must-gather-pzkjr" event={"ID":"28b6f895-fe2f-47b7-926b-4675c409027c","Type":"ContainerStarted","Data":"9feafe9b0f1bdae0eb6048b1486feb3529f749ba2f85d0c0835053fd655844c4"} Jun 08 17:15:21.771217 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:21.771127 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l74s4/must-gather-pzkjr" event={"ID":"28b6f895-fe2f-47b7-926b-4675c409027c","Type":"ContainerStarted","Data":"fc04dae4ffad954d85716080caa3f8f0ad0072b0ba4aa2d8773dd320ca7f57f8"} Jun 08 17:15:21.789164 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:21.789092 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l74s4/must-gather-pzkjr" podStartSLOduration=0.972798216 podStartE2EDuration="1.789069171s" totalImagesPullingTime="816.270955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 17:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 17:15:21.786887243 +0000 UTC m=+2526.173075660" watchObservedRunningTime="2026-06-08 17:15:21.789069171 +0000 UTC m=+2526.175257565" Jun 08 17:15:23.299619 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:23.299578 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lq2k9_93ef7e77-f153-47bb-bd69-2a073996344d/global-pull-secret-syncer/0.log" Jun 08 17:15:23.421551 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:23.421514 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zgjvf_724a94f7-5d35-4fe6-be48-2da68cfa3f54/konnectivity-agent/0.log" Jun 08 17:15:23.499935 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:23.499898 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-92.ec2.internal_c4f5161e243f7f1549af0b115bd8013b/haproxy/0.log" Jun 08 17:15:29.399659 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:29.397927 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-796674c885-7bxq5_e265f4f6-8378-48a6-8766-206f3d1e346a/authorino/0.log" Jun 08 17:15:29.437171 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:29.437133 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rh996_285835d0-dbd4-4fff-a6fe-a7d1f954e297/manager/0.log" Jun 08 17:15:29.539357 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:29.539321 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-m56xq_7766c1c7-0747-4391-af89-7ea28f9d9fb6/registry-server/0.log" Jun 08 17:15:31.732525 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.732450 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-scwpv_bc925f5a-6875-4f48-8be6-e2eca7639413/node-exporter/0.log" Jun 08 17:15:31.762561 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.762519 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-scwpv_bc925f5a-6875-4f48-8be6-e2eca7639413/kube-rbac-proxy/0.log" Jun 08 17:15:31.790345 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.790318 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-scwpv_bc925f5a-6875-4f48-8be6-e2eca7639413/init-textfile/0.log" Jun 08 17:15:31.899492 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.899461 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dv2l8_2938d7ea-6002-4e59-bcb3-eb82515bf947/kube-rbac-proxy-main/0.log" Jun 08 17:15:31.922446 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.922415 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dv2l8_2938d7ea-6002-4e59-bcb3-eb82515bf947/kube-rbac-proxy-self/0.log" Jun 08 17:15:31.947266 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.947227 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dv2l8_2938d7ea-6002-4e59-bcb3-eb82515bf947/openshift-state-metrics/0.log" Jun 08 17:15:31.987129 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:31.987031 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/prometheus/0.log" Jun 08 17:15:32.009763 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.009709 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/config-reloader/0.log" Jun 08 17:15:32.035431 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.035397 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/thanos-sidecar/0.log" Jun 08 17:15:32.058268 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.058225 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/kube-rbac-proxy-web/0.log" Jun 08 17:15:32.082060 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.082021 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/kube-rbac-proxy/0.log" Jun 08 17:15:32.104567 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.104539 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/kube-rbac-proxy-thanos/0.log" Jun 08 17:15:32.130458 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.130426 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e52f2e78-82c6-410b-b523-4cbe18750ed5/init-config-reloader/0.log" Jun 08 17:15:32.171719 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.170095 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-9dgf5_dcc38e97-2f6a-4dbe-baa6-5429bee97124/prometheus-operator/0.log" Jun 08 17:15:32.200860 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.200740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-9dgf5_dcc38e97-2f6a-4dbe-baa6-5429bee97124/kube-rbac-proxy/0.log" Jun 08 17:15:32.266301 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.266215 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867f4f857-5hp5v_64146db8-4329-413f-88d7-3f8d0bac5d8b/telemeter-client/0.log" Jun 08 17:15:32.288033 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.288005 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867f4f857-5hp5v_64146db8-4329-413f-88d7-3f8d0bac5d8b/reload/0.log" Jun 08 17:15:32.310226 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.310189 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867f4f857-5hp5v_64146db8-4329-413f-88d7-3f8d0bac5d8b/kube-rbac-proxy/0.log" Jun 08 17:15:32.343190 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.343156 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/thanos-query/0.log" Jun 08 17:15:32.364721 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.364685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/kube-rbac-proxy-web/0.log" Jun 08 17:15:32.389508 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.389476 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/kube-rbac-proxy/0.log" Jun 08 17:15:32.411547 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.411513 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/prom-label-proxy/0.log" Jun 08 17:15:32.440105 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.440075 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/kube-rbac-proxy-rules/0.log" Jun 08 17:15:32.462809 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:32.462755 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d67f47b99-pbm6b_0e1700cc-2287-4c4b-9c9b-aba2eaf6ee4e/kube-rbac-proxy-metrics/0.log" Jun 08 17:15:33.667859 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:33.667822 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-btgmx_043c7ccf-bec6-49fb-ad22-e51a225045aa/networking-console-plugin/0.log" Jun 08 17:15:35.466972 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.466920 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j"] Jun 08 17:15:35.473004 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.472973 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.480858 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.480821 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j"] Jun 08 17:15:35.543543 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.543156 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-lib-modules\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.543543 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.543230 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-proc\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.543543 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.543347 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-podres\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.543543 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.543377 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-sys\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.543543 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.543408 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkgts\" (UniqueName: \"kubernetes.io/projected/ae056443-207c-43f7-af0b-013f2034b1a0-kube-api-access-pkgts\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644185 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644151 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-podres\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644185 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-sys\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644445 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkgts\" (UniqueName: \"kubernetes.io/projected/ae056443-207c-43f7-af0b-013f2034b1a0-kube-api-access-pkgts\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644445 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644251 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-lib-modules\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644445 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644298 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-proc\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644445 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644344 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-podres\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644445 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644384 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-proc\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644770 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644451 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-sys\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.644770 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.644462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae056443-207c-43f7-af0b-013f2034b1a0-lib-modules\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.652066 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.652036 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkgts\" (UniqueName: \"kubernetes.io/projected/ae056443-207c-43f7-af0b-013f2034b1a0-kube-api-access-pkgts\") pod \"perf-node-gather-daemonset-jbp9j\" (UID: \"ae056443-207c-43f7-af0b-013f2034b1a0\") " pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.789522 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.789421 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:35.969035 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:35.968219 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j"] Jun 08 17:15:36.805050 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.805015 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmgh_a6083b35-bb54-4772-b8ff-93f4bdbab53c/dns/0.log" Jun 08 17:15:36.825970 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.825942 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rbmgh_a6083b35-bb54-4772-b8ff-93f4bdbab53c/kube-rbac-proxy/0.log" Jun 08 17:15:36.871787 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.871759 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kxw8p_1dedd458-09c2-40f1-9dfa-2bd02e1f8965/dns-node-resolver/0.log" Jun 08 17:15:36.887322 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.887289 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" event={"ID":"ae056443-207c-43f7-af0b-013f2034b1a0","Type":"ContainerStarted","Data":"db1033c485ef6a46da84fd76bde4218f92abd03b072d4ad4be7d2a45082e4141"} Jun 08 17:15:36.887503 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.887349 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" event={"ID":"ae056443-207c-43f7-af0b-013f2034b1a0","Type":"ContainerStarted","Data":"b2037e397c077b4ba00eca3479afebd5486d2a34ff4dcc6009365905ed980ca9"} Jun 08 17:15:36.887503 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.887443 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:36.903567 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:36.903513 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" podStartSLOduration=1.903496043 podStartE2EDuration="1.903496043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 17:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 17:15:36.902851543 +0000 UTC m=+2541.289039931" watchObservedRunningTime="2026-06-08 17:15:36.903496043 +0000 UTC m=+2541.289684437" Jun 08 17:15:37.459550 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:37.459522 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bd2x9_40a14650-8507-4ce8-8301-987731013a88/node-ca/0.log" Jun 08 17:15:38.455986 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:38.455957 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795jnn2b_90bf27fa-47f1-4f79-9e73-44f54baf2ec4/istio-proxy/0.log" Jun 08 17:15:38.638587 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:38.638528 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/0.log" Jun 08 17:15:38.705589 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:38.705524 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/1.log" Jun 08 17:15:38.772523 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:38.772429 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-x72c7_ca2483f0-0110-40c0-b699-13404f7be2e0/payload-pre-processing/0.log" Jun 08 17:15:38.891261 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:38.891229 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 17:15:39.507388 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:39.507350 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-72zpv_3b1c4918-0486-4c55-9efa-83db938280a8/serve-healthcheck-canary/0.log" Jun 08 17:15:40.123791 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:40.123758 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc9qq_ee1356b5-bda6-47fb-94d6-e6960d12c53f/kube-rbac-proxy/0.log" Jun 08 17:15:40.145812 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:40.145778 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc9qq_ee1356b5-bda6-47fb-94d6-e6960d12c53f/exporter/0.log" Jun 08 17:15:40.169776 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:40.169742 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cc9qq_ee1356b5-bda6-47fb-94d6-e6960d12c53f/extractor/0.log" Jun 08 17:15:42.444513 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.444418 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-75b7f6c686-8s9lc_5261595f-b064-4015-bada-3748f3fb6fd5/nginx/0.log" Jun 08 17:15:42.715551 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.715475 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29682300-97j2d_2c5df3f0-90ea-4436-8158-2b553bb62918/cleanup/0.log" Jun 08 17:15:42.752867 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.752820 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29682315-7b2b9_8e635853-fdb8-4a11-9080-c4a3fc50e27c/cleanup/0.log" Jun 08 17:15:42.888582 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.888540 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76f9db8d84-tnkdr_e263d322-62d9-4972-9f92-13cc503823de/manager/0.log" Jun 08 17:15:42.906899 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.906869 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l74s4/perf-node-gather-daemonset-jbp9j" Jun 08 17:15:42.921296 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:42.921267 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-2hkjc_2cfc5d20-4fad-42ec-9fc7-3ae3c5c4d139/server/0.log" Jun 08 17:15:43.230621 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:43.230571 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf5fd9654-ht8h8_dfbf5980-9001-430b-abe8-c57cb88c4dc5/manager/0.log" Jun 08 17:15:43.266810 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:43.266781 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf5fd9654-ht8h8_dfbf5980-9001-430b-abe8-c57cb88c4dc5/copy-manifests/0.log" Jun 08 17:15:43.384819 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:43.384782 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-424jq_67fd4edf-95bd-4180-961d-8671251665e8/postgres/0.log" Jun 08 17:15:45.143616 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:45.143581 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b766c7786-hqp8g_c8161c8e-c54c-49ac-8796-f31c339c8746/manager/0.log" Jun 08 17:15:48.777700 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:48.777668 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795jnn2b_90bf27fa-47f1-4f79-9e73-44f54baf2ec4/istio-proxy/0.log" Jun 08 17:15:48.929985 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:48.929951 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/0.log" Jun 08 17:15:48.999338 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:48.999286 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/1.log" Jun 08 17:15:49.052937 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:49.052848 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-x72c7_ca2483f0-0110-40c0-b699-13404f7be2e0/payload-pre-processing/0.log" Jun 08 17:15:49.166121 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:49.166092 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 17:15:50.373810 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:50.373772 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795jnn2b_90bf27fa-47f1-4f79-9e73-44f54baf2ec4/istio-proxy/0.log" Jun 08 17:15:50.531494 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:50.531459 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/0.log" Jun 08 17:15:50.588181 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:50.588148 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/1.log" Jun 08 17:15:50.644604 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:50.644518 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-x72c7_ca2483f0-0110-40c0-b699-13404f7be2e0/payload-pre-processing/0.log" Jun 08 17:15:50.749277 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:50.749250 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 17:15:52.190560 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.190529 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/kube-multus-additional-cni-plugins/0.log" Jun 08 17:15:52.213677 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.213611 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/egress-router-binary-copy/0.log" Jun 08 17:15:52.234412 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.234380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/cni-plugins/0.log" Jun 08 17:15:52.255239 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.255210 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/bond-cni-plugin/0.log" Jun 08 17:15:52.275788 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.275753 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/routeoverride-cni/0.log" Jun 08 17:15:52.296657 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.296600 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/whereabouts-cni-bincopy/0.log" Jun 08 17:15:52.318558 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.318531 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6qkwf_4f962b4e-936c-4cac-a9f3-687fd8bd1c66/whereabouts-cni/0.log" Jun 08 17:15:52.436616 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.436580 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795jnn2b_90bf27fa-47f1-4f79-9e73-44f54baf2ec4/istio-proxy/0.log" Jun 08 17:15:52.584239 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.584145 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/0.log" Jun 08 17:15:52.650135 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.650104 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-h2j8t_1bb94220-a0fa-422c-a75f-cb2cf39cc533/istio-proxy/1.log" Jun 08 17:15:52.715595 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.715558 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-67d7689b65-x72c7_ca2483f0-0110-40c0-b699-13404f7be2e0/payload-pre-processing/0.log" Jun 08 17:15:52.750825 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.750773 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pmwqs_cca56a36-b98a-4080-a137-5aa7848aaca7/kube-multus/0.log" Jun 08 17:15:52.816519 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.816484 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5z57p_f972084f-acf6-4427-b7ea-0f340e15343b/network-metrics-daemon/0.log" Jun 08 17:15:52.823615 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.823591 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-69c5b8477c-6djtn_bc858a75-84d7-4e35-b231-bdc15ef0624f/router/0.log" Jun 08 17:15:52.835546 ip-10-0-141-92 kubenswrapper[2559]: I0608 17:15:52.835471 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5z57p_f972084f-acf6-4427-b7ea-0f340e15343b/kube-rbac-proxy/0.log"