Jul 14 13:25:04.520340 ip-10-0-138-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 14 13:25:04.520349 ip-10-0-138-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 14 13:25:04.520356 ip-10-0-138-114 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 14 13:25:04.520579 ip-10-0-138-114 systemd[1]: Failed to start Kubernetes Kubelet. Jul 14 13:25:14.625895 ip-10-0-138-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 14 13:25:14.625914 ip-10-0-138-114 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8b454c67f82b4963914c0368495b8755 -- Jul 14 13:27:13.406054 ip-10-0-138-114 systemd[1]: Starting Kubernetes Kubelet... Jul 14 13:27:13.839104 ip-10-0-138-114 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:13.839104 ip-10-0-138-114 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 13:27:13.839104 ip-10-0-138-114 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:13.839104 ip-10-0-138-114 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 13:27:13.839104 ip-10-0-138-114 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:27:13.842490 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.842385 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 13:27:13.846849 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846830 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.846849 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846850 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846854 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846858 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846862 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846865 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846868 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846872 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846875 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846878 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846883 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846887 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846890 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846893 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846896 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846899 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846902 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846905 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846907 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846910 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846914 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.846924 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846917 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846920 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846925 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846930 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846933 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846936 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846940 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846943 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846947 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846950 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846953 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846956 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846959 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846963 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846966 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846969 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846972 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846975 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846978 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.847426 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846981 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846984 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846986 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846989 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846992 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846995 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.846998 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847000 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847004 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847007 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847009 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847012 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847015 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847018 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847022 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847025 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847029 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847032 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847035 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.847934 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847037 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847041 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847043 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847046 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847049 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847055 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847059 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847062 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847065 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847068 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847070 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847073 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.847077 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.848418 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848421 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848427 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848432 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848435 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848438 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848442 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848446 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848449 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848453 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848456 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848459 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848462 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848465 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848468 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848471 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848473 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848476 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848479 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848482 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848484 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.848745 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848487 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848490 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848493 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848495 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848498 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848501 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.848503 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849298 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849306 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849309 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.849308 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849312 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849315 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849320 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849322 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849325 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849328 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849331 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849334 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849336 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849339 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849342 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849345 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849348 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849351 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849353 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849356 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849359 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849361 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849364 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849366 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.849565 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849369 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849372 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849375 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849377 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849380 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849383 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849386 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849389 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849392 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849394 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849397 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849400 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849402 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849405 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849409 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849413 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849415 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849418 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849421 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849424 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.850101 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849426 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849429 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849432 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849435 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849437 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849440 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849443 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849446 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849449 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849452 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849455 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849457 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849460 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849463 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849465 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849468 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849471 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849473 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849476 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849478 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.850627 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849482 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849484 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849487 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849490 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849492 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849495 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849499 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849501 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849505 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849510 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849514 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849518 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849521 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849524 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849527 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849533 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849536 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849539 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849542 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.851165 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849545 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849548 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849551 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849554 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849557 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849559 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849562 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849565 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849568 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849571 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849574 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849578 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849582 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849585 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849588 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849591 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.849594 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850098 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850104 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.851645 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850107 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850111 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850122 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850125 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850128 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850131 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850134 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850137 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850140 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850143 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850146 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850148 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850151 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850154 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850157 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850160 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850162 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850166 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850169 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.852145 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850172 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850174 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850177 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850181 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850185 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850188 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850191 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850194 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850196 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850199 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850202 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850205 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850208 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850211 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850214 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850217 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850226 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850229 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850232 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.852629 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850235 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850238 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850240 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850243 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850246 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850249 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850251 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850256 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850260 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850263 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850266 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850269 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850272 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850274 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850277 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850280 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850282 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850285 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850288 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.853157 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850291 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850294 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850296 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850299 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850301 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850304 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850307 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850311 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850314 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850317 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850321 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850330 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850332 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850335 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850338 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850341 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850343 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850346 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850349 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850351 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850354 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.853637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850357 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850359 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850363 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850367 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850370 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850373 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850375 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850378 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850381 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850384 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850386 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850389 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850392 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850394 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850398 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850400 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850403 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850408 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.850411 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850495 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 13:27:13.854187 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850503 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850517 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850522 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850535 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850539 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850544 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850550 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850554 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850557 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850561 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850565 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850568 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850571 2568 flags.go:64] FLAG: --cgroup-root="" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850574 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850578 2568 flags.go:64] FLAG: --client-ca-file="" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850581 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850584 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850592 2568 flags.go:64] FLAG: --cluster-domain="" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850595 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850598 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850602 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850605 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850610 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 13:27:13.854728 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850613 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850616 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850620 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850624 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850627 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850630 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850636 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850639 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850644 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850647 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850650 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850653 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850657 2568 flags.go:64] FLAG: --enable-server="true" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850667 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850675 2568 flags.go:64] FLAG: --event-burst="100" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850678 2568 flags.go:64] FLAG: --event-qps="50" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850681 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850685 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850688 2568 flags.go:64] FLAG: --eviction-hard="" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850692 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850695 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850699 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850702 2568 flags.go:64] FLAG: --eviction-soft="" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850705 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850708 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 13:27:13.855336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850711 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850715 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850718 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850721 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850725 2568 flags.go:64] FLAG: --feature-gates="" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850729 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850732 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850735 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850739 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850743 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850746 2568 flags.go:64] FLAG: --help="false" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850750 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-138-114.ec2.internal" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850753 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850758 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850761 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850765 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850769 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850772 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850775 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850778 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850781 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850791 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850795 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850798 2568 flags.go:64] FLAG: --kube-reserved="" Jul 14 13:27:13.856011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850816 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850826 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850829 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850832 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850836 2568 flags.go:64] FLAG: --lock-file="" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850839 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850842 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850845 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850852 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850855 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850858 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850861 2568 flags.go:64] FLAG: --logging-format="text" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850865 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850868 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850871 2568 flags.go:64] FLAG: --manifest-url="" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850875 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850880 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850883 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850888 2568 flags.go:64] FLAG: --max-pods="110" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850891 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850894 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850899 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850902 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850906 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850909 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 13:27:13.856643 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850912 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850920 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850924 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850927 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850930 2568 flags.go:64] FLAG: --pod-cidr="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850940 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850947 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850950 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850953 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850956 2568 flags.go:64] FLAG: --port="10250" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850959 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850963 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09757c5611781f28e" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850966 2568 flags.go:64] FLAG: --qos-reserved="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850982 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850986 2568 flags.go:64] FLAG: --register-node="true" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850989 2568 flags.go:64] FLAG: --register-with-taints="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850993 2568 flags.go:64] FLAG: --registry-burst="10" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.850997 2568 flags.go:64] FLAG: --registry-qps="5" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851000 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851003 2568 flags.go:64] FLAG: --reserved-memory="" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851007 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851011 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851015 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851021 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851024 2568 flags.go:64] FLAG: --runonce="false" Jul 14 13:27:13.857350 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851027 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851031 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851034 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851039 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851042 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851046 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851049 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851052 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851056 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851059 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851063 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851066 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851069 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851079 2568 flags.go:64] FLAG: --system-cgroups="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851083 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851089 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851092 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851095 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851102 2568 flags.go:64] FLAG: --tls-min-version="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851105 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851108 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851111 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851115 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851118 2568 flags.go:64] FLAG: --v="2" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851123 2568 flags.go:64] FLAG: --version="false" Jul 14 13:27:13.858038 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851127 2568 flags.go:64] FLAG: --vmodule="" Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851131 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.851134 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851310 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851314 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851319 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851324 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851328 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851331 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851334 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851338 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851342 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851345 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851348 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851350 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851353 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851356 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851358 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851361 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851364 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.859227 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851367 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851376 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851379 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851383 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851385 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851388 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851391 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851393 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851396 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851399 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851401 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851404 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851407 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851409 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851412 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851415 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851418 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851422 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851425 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851427 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.860142 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851430 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851433 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851437 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851440 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851443 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851446 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851448 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851451 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851454 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851457 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851459 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851462 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851465 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851467 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851476 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851479 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851482 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851485 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851487 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851490 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.861229 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851493 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851495 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851498 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851500 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851503 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851506 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851509 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851511 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851514 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851519 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851521 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851524 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851527 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851530 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851534 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851537 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851540 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851542 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851545 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.862151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851548 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851552 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851556 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851559 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851563 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851567 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851570 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851573 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851584 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851587 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851590 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851593 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851595 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851598 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851601 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851604 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851606 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851609 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.862697 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851612 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851615 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851617 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851620 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.851624 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.852375 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.860503 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.860529 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.860560 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.860570 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860662 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860670 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860675 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860681 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860688 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.863196 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860693 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860698 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860703 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860708 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860714 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860718 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860723 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860728 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860732 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860736 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860740 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860744 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860748 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860753 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860757 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860761 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860765 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860770 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860775 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860779 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.863603 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860784 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860788 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860792 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860797 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860822 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860827 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860831 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860835 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860839 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860843 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860847 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860852 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860856 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860860 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860873 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860879 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860884 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860892 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860897 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.864150 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860902 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860907 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860911 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860916 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860921 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860926 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860931 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860936 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860941 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860947 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860951 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860955 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860960 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860964 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860969 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860974 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860979 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860984 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860990 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.864637 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860994 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.860999 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861003 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861008 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861015 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861021 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861026 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861031 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861035 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861040 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861044 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861049 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861054 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861058 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861063 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861067 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861072 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861076 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861081 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861086 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.865151 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861091 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861096 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861100 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861105 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861109 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861114 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861119 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861123 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861128 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861132 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861137 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861142 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861147 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861151 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861156 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.866046 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861161 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.861169 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861370 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861380 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861385 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861391 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861396 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861401 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861408 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861415 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861420 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861425 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861430 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861435 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861439 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:27:13.866726 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861444 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861449 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861454 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861458 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861463 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861467 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861472 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861479 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861485 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861490 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861495 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861500 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861505 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861511 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861516 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861522 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861528 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861533 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861538 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:27:13.867199 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861543 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861547 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861552 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861556 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861561 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861565 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861569 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861574 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861578 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861583 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861587 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861592 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861596 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861601 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861605 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861610 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861615 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861620 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861624 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861629 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:27:13.867747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861633 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861637 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861642 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861646 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861651 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861655 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861660 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861664 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861669 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861675 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861680 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861684 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861689 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861693 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861699 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861703 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861707 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861712 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861716 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:27:13.868283 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861722 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861728 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861733 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861737 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861742 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861746 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861751 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861755 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861760 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861765 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861769 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861774 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861778 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861783 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861788 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861793 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861797 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861821 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861827 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861831 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:27:13.868785 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861836 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861840 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861845 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861852 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861856 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861861 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861866 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:13.861870 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.861879 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.862074 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 13:27:13.869373 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.869337 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 13:27:13.871047 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.871033 2568 server.go:1019] "Starting client certificate rotation" Jul 14 13:27:13.871164 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.871143 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:27:13.871217 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.871193 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:27:13.874394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.874374 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:27:13.879414 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.879396 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:27:13.899673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.899645 2568 log.go:25] "Validated CRI v1 runtime API" Jul 14 13:27:13.901977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.901960 2568 log.go:25] "Validated CRI v1 image API" Jul 14 13:27:13.903430 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.903416 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 13:27:13.906014 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.905993 2568 fs.go:135] Filesystem UUIDs: map[5108bd3f-341f-4fc4-91fb-405c128a57ed:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e0b2ed22-e802-44d9-b34c-3d3f5c247f4f:/dev/nvme0n1p3] Jul 14 13:27:13.906060 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.906015 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 13:27:13.913330 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.913199 2568 manager.go:216] Machine: {Timestamp:2026-07-14 13:27:13.911531879 +0000 UTC m=+0.377659450 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099857 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21761b8a12b01b9d81c5724d7c1edb SystemUUID:ec21761b-8a12-b01b-9d81-c5724d7c1edb BootID:8b454c67-f82b-4963-914c-0368495b8755 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:ca:9c:ae:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:ca:9c:ae:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:b4:ea:bb:b0:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 13:27:13.913330 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.913321 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 13:27:13.913458 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.913418 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 13:27:13.915283 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.915247 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 13:27:13.915464 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.915286 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 13:27:13.916024 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.916012 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 13:27:13.916065 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.916026 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 13:27:13.916065 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.916034 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:27:13.916591 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.916579 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:27:13.916675 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.916654 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:27:13.917593 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.917581 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 13:27:13.920418 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920405 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:27:13.920557 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920547 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 13:27:13.920596 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920580 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 14 13:27:13.920596 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920594 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 13:27:13.920660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920619 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 13:27:13.920660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920632 2568 kubelet.go:398] "Adding apiserver pod source" Jul 14 13:27:13.920660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.920642 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 13:27:13.921289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.921259 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:27:13.921289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.921288 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:27:13.923634 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.923595 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 13:27:13.928246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.927470 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 13:27:13.928332 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.928268 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 13:27:13.929003 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.928990 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929008 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929014 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929020 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929030 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929036 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929045 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 13:27:13.929048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929051 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 13:27:13.929246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929058 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 13:27:13.929246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929064 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 13:27:13.929246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929074 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 13:27:13.929246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.929079 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 13:27:13.930034 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.930025 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 13:27:13.930034 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.930034 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 13:27:13.933887 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.933839 2568 server.go:1268] "Started kubelet" Jul 14 13:27:13.933989 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.933935 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 13:27:13.934058 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.934012 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 13:27:13.934176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.934148 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 13:27:13.934466 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.934437 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 13:27:13.934568 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.934433 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 13:27:13.934707 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.934677 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 13:27:13.934989 ip-10-0-138-114 systemd[1]: Started Kubernetes Kubelet. Jul 14 13:27:13.935287 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.935266 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 13:27:13.936904 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.936889 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 14 13:27:13.940252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.940231 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 13:27:13.940252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.940249 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 13:27:13.940387 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.940342 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:13.940918 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.940893 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 13:27:13.940918 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.940913 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 13:27:13.941067 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.940925 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 13:27:13.941173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941158 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 14 13:27:13.941272 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941262 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 14 13:27:13.941476 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941454 2568 factory.go:55] Registering systemd factory Jul 14 13:27:13.941534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941478 2568 factory.go:223] Registration of the systemd container factory successfully Jul 14 13:27:13.941812 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941791 2568 factory.go:153] Registering CRI-O factory Jul 14 13:27:13.941910 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941834 2568 factory.go:223] Registration of the crio container factory successfully Jul 14 13:27:13.941910 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941895 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 13:27:13.942036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941921 2568 factory.go:103] Registering Raw factory Jul 14 13:27:13.942036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.941950 2568 manager.go:1196] Started watching for new ooms in manager Jul 14 13:27:13.943903 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.942679 2568 manager.go:349] Starting recovery of all containers Jul 14 13:27:13.944340 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.944279 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 13:27:13.950899 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.950873 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 13:27:13.950899 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.950892 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 13:27:13.956052 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.951037 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-114.ec2.internal.18c22af4f343336e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-114.ec2.internal,UID:ip-10-0-138-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-114.ec2.internal,},FirstTimestamp:2026-07-14 13:27:13.93379211 +0000 UTC m=+0.399919659,LastTimestamp:2026-07-14 13:27:13.93379211 +0000 UTC m=+0.399919659,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-114.ec2.internal,}" Jul 14 13:27:13.958011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.957968 2568 manager.go:354] Recovery completed Jul 14 13:27:13.963951 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.963936 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:13.966128 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966111 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:13.966198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966142 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:13.966198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966154 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:13.966745 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966718 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 13:27:13.966745 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966745 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 13:27:13.966858 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966762 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:27:13.966858 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.966724 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:13.967594 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967578 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:13.967654 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967604 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:13.967654 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967615 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:13.967904 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967893 2568 policy_none.go:49] "None policy: Start" Jul 14 13:27:13.967939 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967911 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 13:27:13.967939 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.967922 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 13:27:13.968907 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.968895 2568 policy_none.go:47] "Start" Jul 14 13:27:13.971394 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.971323 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-114.ec2.internal.18c22af4f5309608 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-114.ec2.internal,UID:ip-10-0-138-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-114.ec2.internal,},FirstTimestamp:2026-07-14 13:27:13.9661266 +0000 UTC m=+0.432254147,LastTimestamp:2026-07-14 13:27:13.9661266 +0000 UTC m=+0.432254147,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-114.ec2.internal,}" Jul 14 13:27:13.986116 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:13.986025 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-114.ec2.internal.18c22af4f530e7cf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-114.ec2.internal,UID:ip-10-0-138-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-114.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-114.ec2.internal,},FirstTimestamp:2026-07-14 13:27:13.966147535 +0000 UTC m=+0.432275082,LastTimestamp:2026-07-14 13:27:13.966147535 +0000 UTC m=+0.432275082,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-114.ec2.internal,}" Jul 14 13:27:13.997900 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:13.997871 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwzlx" Jul 14 13:27:14.003452 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.003423 2568 manager.go:337] "Starting Device Plugin manager" Jul 14 13:27:14.003567 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.003490 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 13:27:14.003567 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.003505 2568 server.go:87] "Starting device plugin registration server" Jul 14 13:27:14.003659 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.003534 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-114.ec2.internal.18c22af4f531120a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-114.ec2.internal,UID:ip-10-0-138-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-114.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-114.ec2.internal,},FirstTimestamp:2026-07-14 13:27:13.966158346 +0000 UTC m=+0.432285893,LastTimestamp:2026-07-14 13:27:13.966158346 +0000 UTC m=+0.432285893,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-114.ec2.internal,}" Jul 14 13:27:14.003864 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.003849 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 13:27:14.003922 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.003866 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 13:27:14.003964 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.003952 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 13:27:14.004068 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.004052 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 13:27:14.004117 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.004072 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 13:27:14.004720 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.004699 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 13:27:14.004833 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.004776 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.008550 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.008533 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zwzlx" Jul 14 13:27:14.014303 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.014225 2568 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-138-114.ec2.internal.18c22af4f5309608\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-114.ec2.internal.18c22af4f5309608 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-114.ec2.internal,UID:ip-10-0-138-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-114.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-114.ec2.internal,},FirstTimestamp:2026-07-14 13:27:13.9661266 +0000 UTC m=+0.432254147,LastTimestamp:2026-07-14 13:27:13.967592165 +0000 UTC m=+0.433719715,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-114.ec2.internal,}" Jul 14 13:27:14.105107 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.105067 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:14.106339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.106208 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:14.106339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.106238 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:14.106339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.106252 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:14.106339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.106288 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.117148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.117127 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.117200 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.117152 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-114.ec2.internal\": node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.129714 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.129668 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 13:27:14.130983 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.130964 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 13:27:14.131100 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.131003 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 13:27:14.131100 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.131023 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 13:27:14.131100 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.131056 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 13:27:14.136607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.136585 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:14.156886 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.156855 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.231996 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.231908 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal"] Jul 14 13:27:14.232144 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.232009 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:14.233108 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.233089 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:14.233220 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.233123 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:14.233220 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.233133 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:14.234495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.234480 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:14.234624 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.234609 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.234667 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.234641 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:14.235365 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235350 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:14.235443 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235371 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:14.235443 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235381 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:14.235792 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235756 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:14.235878 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235807 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:14.235878 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.235823 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:14.236744 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.236715 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.236828 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.236754 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:14.243278 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.243259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.244213 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.244194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.244275 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.244225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6d51354587a329354b548be5d6cdd37-config\") pod \"kube-apiserver-proxy-ip-10-0-138-114.ec2.internal\" (UID: \"c6d51354587a329354b548be5d6cdd37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.245156 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.245142 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:14.245228 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.245169 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:14.245228 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.245181 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:14.256503 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.256474 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-114.ec2.internal\" not found" node="ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.257640 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.257625 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.263300 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.263281 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-114.ec2.internal\" not found" node="ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345243 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345207 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345243 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345441 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6d51354587a329354b548be5d6cdd37-config\") pod \"kube-apiserver-proxy-ip-10-0-138-114.ec2.internal\" (UID: \"c6d51354587a329354b548be5d6cdd37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345441 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345441 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb94a58b8790d4f4e68607b8aa46901b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal\" (UID: \"cb94a58b8790d4f4e68607b8aa46901b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.345441 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.345341 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c6d51354587a329354b548be5d6cdd37-config\") pod \"kube-apiserver-proxy-ip-10-0-138-114.ec2.internal\" (UID: \"c6d51354587a329354b548be5d6cdd37\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.358014 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.357989 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.458212 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.458170 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.558845 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.558769 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.559917 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.559879 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.566148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.566125 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:14.659138 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.659088 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.759818 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.759776 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.858182 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.858106 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:14.860322 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.860304 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:14.870036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.870013 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 13:27:14.870203 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.870182 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 13:27:14.870239 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:14.870208 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 13:27:14.961470 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:14.961442 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:15.004893 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.004845 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:27:15.006034 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.006016 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:27:15.006129 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.006047 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:27:15.006129 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.006059 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:27:15.013618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.013586 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 13:22:14 +0000 UTC" deadline="2028-01-11 10:42:25.695845722 +0000 UTC" Jul 14 13:27:15.013672 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.013619 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13101h15m10.682229613s" Jul 14 13:27:15.061960 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.061925 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:15.162970 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.162877 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:15.263024 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.262982 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:15.363111 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.363081 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-114.ec2.internal\" not found" Jul 14 13:27:15.366661 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.366641 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:15.398213 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.398180 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:15.440564 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.440477 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" Jul 14 13:27:15.466933 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.466911 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:27:15.468503 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.468488 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" Jul 14 13:27:15.475696 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:15.475669 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d51354587a329354b548be5d6cdd37.slice/crio-9b16167ad77aeb7b5a11ca39a71845f173dc2068c5f243f8ffcaf59c212e82e8 WatchSource:0}: Error finding container 9b16167ad77aeb7b5a11ca39a71845f173dc2068c5f243f8ffcaf59c212e82e8: Status 404 returned error can't find the container with id 9b16167ad77aeb7b5a11ca39a71845f173dc2068c5f243f8ffcaf59c212e82e8 Jul 14 13:27:15.484411 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.484389 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:27:15.489289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.489267 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:27:15.491133 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:15.491111 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb94a58b8790d4f4e68607b8aa46901b.slice/crio-8caa6964dfe2622b8bda222f993f1196af60f8201fdd04fa2da5c51593f6aa2a WatchSource:0}: Error finding container 8caa6964dfe2622b8bda222f993f1196af60f8201fdd04fa2da5c51593f6aa2a: Status 404 returned error can't find the container with id 8caa6964dfe2622b8bda222f993f1196af60f8201fdd04fa2da5c51593f6aa2a Jul 14 13:27:15.924444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.924357 2568 apiserver.go:52] "Watching apiserver" Jul 14 13:27:15.933222 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.933204 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 13:27:15.934537 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.934515 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-vkdmr","openshift-multus/network-metrics-daemon-sdbxt","openshift-network-diagnostics/network-check-target-gzfsj","openshift-network-operator/iptables-alerter-zpsd5","kube-system/konnectivity-agent-r5cbh","openshift-cluster-node-tuning-operator/tuned-62bn9","openshift-dns/node-resolver-gdwtm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal","openshift-multus/multus-additional-cni-plugins-nbgng","openshift-ovn-kubernetes/ovnkube-node-b6mvv","kube-system/global-pull-secret-syncer-krb8t","kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p","openshift-image-registry/node-ca-t8wfc"] Jul 14 13:27:15.936710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.936690 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.938220 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.938201 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:15.938307 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.938261 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:15.940121 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.940099 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.941540 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.941522 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 13:27:15.941834 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.941797 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.943903 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.943886 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:15.945775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.945757 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 13:27:15.945893 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.945843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 13:27:15.945963 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.945909 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.945963 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.945915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:15.946071 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.945968 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 13:27:15.946071 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.946009 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.946071 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.946022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-25p99\"" Jul 14 13:27:15.947861 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.947843 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:15.950012 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.949998 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:15.952258 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.952243 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:15.952342 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.952295 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:15.954321 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954303 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-slash\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954399 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-bin\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954399 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954347 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:15.954399 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954364 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-netd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954516 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:15.954406 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:15.954516 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954516 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-socket-dir-parent\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.954516 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-multus\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.954516 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954505 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-os-release\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954554 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954574 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2tx\" (UniqueName: \"kubernetes.io/projected/137eec1b-b24f-4c5d-9eb7-81601d40c532-kube-api-access-hd2tx\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954589 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-kubelet\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954637 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-conf-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-log-socket\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.954715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-binary-copy\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954761 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b59qj\" (UniqueName: \"kubernetes.io/projected/581c0fd4-fdfb-44fd-992c-f8728532e277-kube-api-access-b59qj\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-systemd-units\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-netns\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-etc-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954900 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-ovn\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954959 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-config\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovn-node-metrics-cert\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.954999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cni-binary-copy\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-etc-kubernetes\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955132 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-var-lib-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cnibin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-k8s-cni-cncf-io\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-bin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-multus-certs\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-cnibin\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-systemd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-env-overrides\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955381 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-os-release\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-netns\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8h8m\" (UniqueName: \"kubernetes.io/projected/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-kube-api-access-b8h8m\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955471 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-node-log\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-kubelet\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.955555 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-script-lib\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955575 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t7wq\" (UniqueName: \"kubernetes.io/projected/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-kube-api-access-7t7wq\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-system-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955623 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-daemon-config\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955676 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-hostroot\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-system-cni-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.956193 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.955759 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:15.957572 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.957366 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:15.959618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.959593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:15.961822 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.961796 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.963042 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.963022 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.963231 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.963212 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 13:27:15.963893 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.963871 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 13:27:15.964272 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xh88p\"" Jul 14 13:27:15.964272 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964227 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.964272 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964256 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.964464 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964326 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ts6z6\"" Jul 14 13:27:15.964464 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964254 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.965015 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.964993 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kj27l\"" Jul 14 13:27:15.965015 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965009 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fwrqk\"" Jul 14 13:27:15.965173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965002 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.965173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965043 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 13:27:15.965173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965061 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 13:27:15.965173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965097 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.965374 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965241 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 13:27:15.965374 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.965343 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jz5kp\"" Jul 14 13:27:15.969171 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.969150 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.970626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.970610 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bksxw\"" Jul 14 13:27:15.974723 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.974705 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.974842 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.974730 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 13:27:15.975057 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.975043 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r5cv6\"" Jul 14 13:27:15.981109 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.981088 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 13:27:15.981418 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.981356 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cstbr\"" Jul 14 13:27:15.981634 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.981613 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 13:27:15.981739 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.981644 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 13:27:15.981998 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.981980 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 13:27:15.982076 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.982008 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 13:27:15.986123 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.986102 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 13:27:15.987021 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:15.986997 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 13:27:16.042867 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.042830 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 13:27:16.049115 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.049086 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:27:16.056932 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.056906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-hostroot\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.056945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.056965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.056991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-kubernetes\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-system-cni-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.057059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057050 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-hostroot\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-bin\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057115 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-netd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057130 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-system-cni-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-bin\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-multus\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-cni-netd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-kubelet\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057219 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nb6\" (UniqueName: \"kubernetes.io/projected/389763af-1d4f-4f88-8956-0c53b42c720f-kube-api-access-w2nb6\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057245 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-multus\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057268 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-kubelet\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057294 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-sys\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057312 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-modprobe-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057334 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-var-lib-kubelet\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.057386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-binary-copy\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-systemd-units\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057458 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-systemd-units\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057495 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-netns\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-etc-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057531 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-netns\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-ovn\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-etc-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057585 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-serviceca\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-agent-certs\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-ovn\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-config\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057679 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovn-node-metrics-cert\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-etc-kubernetes\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.058219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-etc-kubernetes\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-conf\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057887 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-systemd\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-lib-modules\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.057959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-var-lib-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058009 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-var-lib-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-k8s-cni-cncf-io\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058067 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-binary-copy\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-dbus\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-k8s-cni-cncf-io\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-host\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058143 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-systemd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-netns\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8h8m\" (UniqueName: \"kubernetes.io/projected/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-kube-api-access-b8h8m\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058283 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:16.059044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-systemd\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058297 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-netns\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t7wq\" (UniqueName: \"kubernetes.io/projected/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-kube-api-access-7t7wq\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-system-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-device-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.058397 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-host\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058431 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-iptables-alerter-script\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-config\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.058464 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:16.558442491 +0000 UTC m=+3.024570049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-system-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058515 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-slash\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058581 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-socket-dir-parent\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-conf-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058636 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b30953bd-322a-4251-b5c4-1fb7799a8384-hosts-file\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058655 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-slash\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.059920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b30953bd-322a-4251-b5c4-1fb7799a8384-tmp-dir\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fvtz\" (UniqueName: \"kubernetes.io/projected/b30953bd-322a-4251-b5c4-1fb7799a8384-kube-api-access-5fvtz\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058717 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-tmp\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058702 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-socket-dir-parent\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-os-release\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058719 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-conf-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058829 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-os-release\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2tx\" (UniqueName: \"kubernetes.io/projected/137eec1b-b24f-4c5d-9eb7-81601d40c532-kube-api-access-hd2tx\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs2hv\" (UniqueName: \"kubernetes.io/projected/3785a337-66d1-4949-8d42-deb5d4008646-kube-api-access-hs2hv\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-konnectivity-ca\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.058991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-log-socket\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059000 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.060775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059050 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059018 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b59qj\" (UniqueName: \"kubernetes.io/projected/581c0fd4-fdfb-44fd-992c-f8728532e277-kube-api-access-b59qj\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059099 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-run-openvswitch\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysconfig\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-etc-tuned\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cni-binary-copy\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-kubelet-config\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m54bd\" (UniqueName: \"kubernetes.io/projected/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-kube-api-access-m54bd\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059279 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-log-socket\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cnibin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059336 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/581c0fd4-fdfb-44fd-992c-f8728532e277-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-bin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059377 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-multus-certs\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059387 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cnibin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059415 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq7gg\" (UniqueName: \"kubernetes.io/projected/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-kube-api-access-qq7gg\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.061600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-cnibin\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059469 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-cni-dir\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059471 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-env-overrides\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059516 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-var-lib-cni-bin\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-host-run-multus-certs\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059562 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/581c0fd4-fdfb-44fd-992c-f8728532e277-cnibin\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-os-release\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-node-log\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-os-release\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-kubelet\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059718 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-node-log\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-script-lib\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-daemon-config\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059942 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-run\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.059975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-host-slash\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.060364 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-env-overrides\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.060369 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-host-kubelet\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.062561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.060509 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-multus-daemon-config\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.063491 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.060952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-cni-binary-copy\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.063491 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.061386 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovnkube-script-lib\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.063491 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.061921 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-ovn-node-metrics-cert\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.065512 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.065478 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fnvg" Jul 14 13:27:16.087113 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.087091 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2fnvg" Jul 14 13:27:16.093430 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.093405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t7wq\" (UniqueName: \"kubernetes.io/projected/745ddaf3-baf3-4b8a-b12b-cea1b04446e7-kube-api-access-7t7wq\") pod \"ovnkube-node-b6mvv\" (UID: \"745ddaf3-baf3-4b8a-b12b-cea1b04446e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.101734 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.101710 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8h8m\" (UniqueName: \"kubernetes.io/projected/0fefb8f4-1dd4-4d15-aa77-bb897fa3528c-kube-api-access-b8h8m\") pod \"multus-vkdmr\" (UID: \"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c\") " pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.107525 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.107501 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2tx\" (UniqueName: \"kubernetes.io/projected/137eec1b-b24f-4c5d-9eb7-81601d40c532-kube-api-access-hd2tx\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:16.108339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.108321 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b59qj\" (UniqueName: \"kubernetes.io/projected/581c0fd4-fdfb-44fd-992c-f8728532e277-kube-api-access-b59qj\") pod \"multus-additional-cni-plugins-nbgng\" (UID: \"581c0fd4-fdfb-44fd-992c-f8728532e277\") " pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.132428 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.132401 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:16.136011 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.135949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" event={"ID":"cb94a58b8790d4f4e68607b8aa46901b","Type":"ContainerStarted","Data":"8caa6964dfe2622b8bda222f993f1196af60f8201fdd04fa2da5c51593f6aa2a"} Jul 14 13:27:16.137151 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.137129 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" event={"ID":"c6d51354587a329354b548be5d6cdd37","Type":"ContainerStarted","Data":"9b16167ad77aeb7b5a11ca39a71845f173dc2068c5f243f8ffcaf59c212e82e8"} Jul 14 13:27:16.160333 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160298 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b30953bd-322a-4251-b5c4-1fb7799a8384-tmp-dir\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.160465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fvtz\" (UniqueName: \"kubernetes.io/projected/b30953bd-322a-4251-b5c4-1fb7799a8384-kube-api-access-5fvtz\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.160465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-tmp\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.160465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.160465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs2hv\" (UniqueName: \"kubernetes.io/projected/3785a337-66d1-4949-8d42-deb5d4008646-kube-api-access-hs2hv\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-konnectivity-ca\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysconfig\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160555 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-etc-tuned\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160579 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-kubelet-config\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.160597 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160695 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b30953bd-322a-4251-b5c4-1fb7799a8384-tmp-dir\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.160710 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:16.660656492 +0000 UTC m=+3.126784038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160702 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysconfig\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.160742 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m54bd\" (UniqueName: \"kubernetes.io/projected/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-kube-api-access-m54bd\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160812 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-kubelet-config\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq7gg\" (UniqueName: \"kubernetes.io/projected/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-kube-api-access-qq7gg\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160881 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-sys-fs\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160895 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-run\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160939 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-run\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.160977 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161022 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-host-slash\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161107 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161131 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-host-slash\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161133 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-kubernetes\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161175 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-registration-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161183 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-kubernetes\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161185 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nb6\" (UniqueName: \"kubernetes.io/projected/389763af-1d4f-4f88-8956-0c53b42c720f-kube-api-access-w2nb6\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.161299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161209 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-socket-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-sys\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-modprobe-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-var-lib-kubelet\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161278 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-sys\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161301 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-serviceca\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-agent-certs\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161374 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161395 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-modprobe-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-conf\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161438 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-systemd\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161446 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-konnectivity-ca\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161465 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-lib-modules\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-d\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-dbus\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-var-lib-kubelet\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:16.162176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-host\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161575 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-lib-modules\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d139e885-1141-43fc-a857-62e336ce02c5-dbus\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161618 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-device-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-host\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-systemd\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-host\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161722 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-serviceca\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161745 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-etc-sysctl-conf\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-iptables-alerter-script\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/389763af-1d4f-4f88-8956-0c53b42c720f-device-dir\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b30953bd-322a-4251-b5c4-1fb7799a8384-hosts-file\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161862 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3785a337-66d1-4949-8d42-deb5d4008646-host\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.161976 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b30953bd-322a-4251-b5c4-1fb7799a8384-hosts-file\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.163148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.162350 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-iptables-alerter-script\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.163775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.163530 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-etc-tuned\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.164030 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.164007 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3785a337-66d1-4949-8d42-deb5d4008646-tmp\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.164483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.164463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cfe8616-0e6e-43a0-86f3-74dbdaf03607-agent-certs\") pod \"konnectivity-agent-r5cbh\" (UID: \"9cfe8616-0e6e-43a0-86f3-74dbdaf03607\") " pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.181060 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.180969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fvtz\" (UniqueName: \"kubernetes.io/projected/b30953bd-322a-4251-b5c4-1fb7799a8384-kube-api-access-5fvtz\") pod \"node-resolver-gdwtm\" (UID: \"b30953bd-322a-4251-b5c4-1fb7799a8384\") " pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.181599 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.181563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m54bd\" (UniqueName: \"kubernetes.io/projected/b4df8cf1-1b78-4cb1-a6b3-d0239d090817-kube-api-access-m54bd\") pod \"node-ca-t8wfc\" (UID: \"b4df8cf1-1b78-4cb1-a6b3-d0239d090817\") " pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.185930 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.185903 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs2hv\" (UniqueName: \"kubernetes.io/projected/3785a337-66d1-4949-8d42-deb5d4008646-kube-api-access-hs2hv\") pod \"tuned-62bn9\" (UID: \"3785a337-66d1-4949-8d42-deb5d4008646\") " pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.188358 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.188329 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq7gg\" (UniqueName: \"kubernetes.io/projected/121c8fbe-b4dd-42ff-8dba-798ef736dfc2-kube-api-access-qq7gg\") pod \"iptables-alerter-zpsd5\" (UID: \"121c8fbe-b4dd-42ff-8dba-798ef736dfc2\") " pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.190501 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.190480 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:16.190590 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.190507 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:16.190590 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.190522 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:16.190590 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.190586 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:16.690566963 +0000 UTC m=+3.156694512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:16.193447 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.193424 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nb6\" (UniqueName: \"kubernetes.io/projected/389763af-1d4f-4f88-8956-0c53b42c720f-kube-api-access-w2nb6\") pod \"aws-ebs-csi-driver-node-k6l7p\" (UID: \"389763af-1d4f-4f88-8956-0c53b42c720f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.246908 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.246873 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nbgng" Jul 14 13:27:16.252673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.252650 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:16.260969 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.260438 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581c0fd4_fdfb_44fd_992c_f8728532e277.slice/crio-28a17dd6a4e01e19e92f4e44ec016947ee9aba182c4b248592f94b4f4caa5756 WatchSource:0}: Error finding container 28a17dd6a4e01e19e92f4e44ec016947ee9aba182c4b248592f94b4f4caa5756: Status 404 returned error can't find the container with id 28a17dd6a4e01e19e92f4e44ec016947ee9aba182c4b248592f94b4f4caa5756 Jul 14 13:27:16.265301 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.264952 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkdmr" Jul 14 13:27:16.270444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.270346 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:16.277582 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.277556 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fefb8f4_1dd4_4d15_aa77_bb897fa3528c.slice/crio-cb2a1558664cf39575423f6e0cd174ee533966a40414c1a95b514cdbf2e52b73 WatchSource:0}: Error finding container cb2a1558664cf39575423f6e0cd174ee533966a40414c1a95b514cdbf2e52b73: Status 404 returned error can't find the container with id cb2a1558664cf39575423f6e0cd174ee533966a40414c1a95b514cdbf2e52b73 Jul 14 13:27:16.278124 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.278042 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zpsd5" Jul 14 13:27:16.282044 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.281993 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfe8616_0e6e_43a0_86f3_74dbdaf03607.slice/crio-e5c3b256ff8760532aa830b7a1eaa74ee8582b445143a8425e79faa9127d3d66 WatchSource:0}: Error finding container e5c3b256ff8760532aa830b7a1eaa74ee8582b445143a8425e79faa9127d3d66: Status 404 returned error can't find the container with id e5c3b256ff8760532aa830b7a1eaa74ee8582b445143a8425e79faa9127d3d66 Jul 14 13:27:16.286383 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.286362 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdwtm" Jul 14 13:27:16.291378 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.291342 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121c8fbe_b4dd_42ff_8dba_798ef736dfc2.slice/crio-b3d1a154c2764ff2545e39128e7863435d754b8f277aa2930381e90fd3a02bca WatchSource:0}: Error finding container b3d1a154c2764ff2545e39128e7863435d754b8f277aa2930381e90fd3a02bca: Status 404 returned error can't find the container with id b3d1a154c2764ff2545e39128e7863435d754b8f277aa2930381e90fd3a02bca Jul 14 13:27:16.295230 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.295207 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62bn9" Jul 14 13:27:16.300401 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.300373 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30953bd_322a_4251_b5c4_1fb7799a8384.slice/crio-326cd631bc8c8067de109b1e178e902d11efdeed0a7bdcfc36b94c0033c45305 WatchSource:0}: Error finding container 326cd631bc8c8067de109b1e178e902d11efdeed0a7bdcfc36b94c0033c45305: Status 404 returned error can't find the container with id 326cd631bc8c8067de109b1e178e902d11efdeed0a7bdcfc36b94c0033c45305 Jul 14 13:27:16.301607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.301450 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" Jul 14 13:27:16.307047 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.307026 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t8wfc" Jul 14 13:27:16.311965 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.311934 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3785a337_66d1_4949_8d42_deb5d4008646.slice/crio-f8f0a32153ea21372d8f3746cf48dddb43b61b3755d0242e59b8d2d2cf18105e WatchSource:0}: Error finding container f8f0a32153ea21372d8f3746cf48dddb43b61b3755d0242e59b8d2d2cf18105e: Status 404 returned error can't find the container with id f8f0a32153ea21372d8f3746cf48dddb43b61b3755d0242e59b8d2d2cf18105e Jul 14 13:27:16.316585 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.316560 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389763af_1d4f_4f88_8956_0c53b42c720f.slice/crio-ee4e4cd3107454a88bac03b438ba37b458de2dad6bce14d07a5009532fa93d39 WatchSource:0}: Error finding container ee4e4cd3107454a88bac03b438ba37b458de2dad6bce14d07a5009532fa93d39: Status 404 returned error can't find the container with id ee4e4cd3107454a88bac03b438ba37b458de2dad6bce14d07a5009532fa93d39 Jul 14 13:27:16.320074 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:16.320023 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4df8cf1_1b78_4cb1_a6b3_d0239d090817.slice/crio-1592ac8423bcf106f7f912ae0fe9bcf3af98b923376ccc7127009555772e2c0e WatchSource:0}: Error finding container 1592ac8423bcf106f7f912ae0fe9bcf3af98b923376ccc7127009555772e2c0e: Status 404 returned error can't find the container with id 1592ac8423bcf106f7f912ae0fe9bcf3af98b923376ccc7127009555772e2c0e Jul 14 13:27:16.564611 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.564509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:16.564948 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.564921 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:16.565041 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.565022 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:17.5649999 +0000 UTC m=+4.031127453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:16.665348 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.665309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:16.665554 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.665526 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:16.665679 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.665603 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:17.66558419 +0000 UTC m=+4.131711729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:16.766859 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:16.766813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:16.767060 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.767036 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:16.767153 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.767070 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:16.767153 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.767087 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:16.767258 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:16.767158 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:17.767136171 +0000 UTC m=+4.233263721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:17.091249 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.091145 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:22:16 +0000 UTC" deadline="2028-04-10 10:57:07.619034778 +0000 UTC" Jul 14 13:27:17.091249 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.091211 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15261h29m50.527828325s" Jul 14 13:27:17.132025 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.131987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:17.132208 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.132142 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:17.132581 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.132558 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:17.132681 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.132658 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:17.141175 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.141094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8wfc" event={"ID":"b4df8cf1-1b78-4cb1-a6b3-d0239d090817","Type":"ContainerStarted","Data":"1592ac8423bcf106f7f912ae0fe9bcf3af98b923376ccc7127009555772e2c0e"} Jul 14 13:27:17.142956 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.142925 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" event={"ID":"389763af-1d4f-4f88-8956-0c53b42c720f","Type":"ContainerStarted","Data":"ee4e4cd3107454a88bac03b438ba37b458de2dad6bce14d07a5009532fa93d39"} Jul 14 13:27:17.144480 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.144456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdwtm" event={"ID":"b30953bd-322a-4251-b5c4-1fb7799a8384","Type":"ContainerStarted","Data":"326cd631bc8c8067de109b1e178e902d11efdeed0a7bdcfc36b94c0033c45305"} Jul 14 13:27:17.146297 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.146244 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zpsd5" event={"ID":"121c8fbe-b4dd-42ff-8dba-798ef736dfc2","Type":"ContainerStarted","Data":"b3d1a154c2764ff2545e39128e7863435d754b8f277aa2930381e90fd3a02bca"} Jul 14 13:27:17.148060 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.148034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r5cbh" event={"ID":"9cfe8616-0e6e-43a0-86f3-74dbdaf03607","Type":"ContainerStarted","Data":"e5c3b256ff8760532aa830b7a1eaa74ee8582b445143a8425e79faa9127d3d66"} Jul 14 13:27:17.149541 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.149515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"107f410687f31f087338115bcc3ac1cc914b172a2c6f4cf21e9c4b42c6fa4bbe"} Jul 14 13:27:17.151323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.151301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62bn9" event={"ID":"3785a337-66d1-4949-8d42-deb5d4008646","Type":"ContainerStarted","Data":"f8f0a32153ea21372d8f3746cf48dddb43b61b3755d0242e59b8d2d2cf18105e"} Jul 14 13:27:17.153214 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.153191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkdmr" event={"ID":"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c","Type":"ContainerStarted","Data":"cb2a1558664cf39575423f6e0cd174ee533966a40414c1a95b514cdbf2e52b73"} Jul 14 13:27:17.154777 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.154755 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerStarted","Data":"28a17dd6a4e01e19e92f4e44ec016947ee9aba182c4b248592f94b4f4caa5756"} Jul 14 13:27:17.573970 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.573880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:17.574150 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.574077 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:17.574212 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.574150 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:19.574128108 +0000 UTC m=+6.040255667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:17.674698 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.674658 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:17.674885 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.674867 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:17.674967 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.674944 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:19.67492224 +0000 UTC m=+6.141049790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:17.736091 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.736048 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:27:17.776372 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:17.775968 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:17.776372 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.776148 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:17.776372 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.776175 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:17.776372 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.776189 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:17.776372 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:17.776253 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:19.776233017 +0000 UTC m=+6.242360551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:18.093710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:18.093474 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:22:16 +0000 UTC" deadline="2028-02-19 20:56:15.444462311 +0000 UTC" Jul 14 13:27:18.093710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:18.093710 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14047h28m57.350759187s" Jul 14 13:27:18.133986 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:18.133929 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:18.134104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:18.134055 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:18.179901 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:18.179772 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" event={"ID":"c6d51354587a329354b548be5d6cdd37","Type":"ContainerStarted","Data":"75330c71427fb931db121785c121009cfcb92521cdd47109a5ad4f151d78fe54"} Jul 14 13:27:18.205018 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:18.204932 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-114.ec2.internal" podStartSLOduration=2.204906646 podStartE2EDuration="2.204906646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:18.203786709 +0000 UTC m=+4.669914290" watchObservedRunningTime="2026-07-14 13:27:18.204906646 +0000 UTC m=+4.671034203" Jul 14 13:27:19.131948 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.131911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:19.132447 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.131911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:19.132447 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.132152 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:19.132447 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.132322 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:19.184682 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.184642 2568 generic.go:356] "Generic (PLEG): container finished" podID="cb94a58b8790d4f4e68607b8aa46901b" containerID="0ecfece17efd33fd939f4d0bf6f7f3115925e637bd0dde93ba1dd923e343478c" exitCode=0 Jul 14 13:27:19.184997 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.184968 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" event={"ID":"cb94a58b8790d4f4e68607b8aa46901b","Type":"ContainerDied","Data":"0ecfece17efd33fd939f4d0bf6f7f3115925e637bd0dde93ba1dd923e343478c"} Jul 14 13:27:19.594982 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.594943 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:19.595169 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.595134 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:19.595251 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.595197 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:23.595176142 +0000 UTC m=+10.061303706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:19.696053 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.696012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:19.696238 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.696207 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:19.696302 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.696267 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:23.696248736 +0000 UTC m=+10.162376288 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:19.796940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:19.796905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:19.797162 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.797143 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:19.797226 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.797169 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:19.797226 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.797184 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:19.797342 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:19.797241 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:23.797222179 +0000 UTC m=+10.263349718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:20.137143 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:20.137105 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:20.137640 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:20.137524 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:21.131265 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:21.131231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:21.131446 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:21.131395 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:21.131446 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:21.131413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:21.131694 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:21.131530 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:22.131656 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:22.131391 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:22.131656 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:22.131548 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:23.131326 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:23.131289 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:23.131514 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.131436 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:23.131514 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:23.131497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:23.131663 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.131628 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:23.631704 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:23.631216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:23.631704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.631391 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:23.631704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.631455 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:31.631436363 +0000 UTC m=+18.097563939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:23.731859 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:23.731815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:23.732118 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.732080 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:23.732213 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.732170 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:31.732149077 +0000 UTC m=+18.198276611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:23.833086 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:23.833047 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:23.833281 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.833264 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:23.833342 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.833285 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:23.833342 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.833298 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:23.833444 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:23.833356 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:31.833338787 +0000 UTC m=+18.299466323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:24.132575 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:24.132544 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:24.132811 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:24.132647 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:25.132240 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:25.132202 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:25.132240 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:25.132229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:25.132842 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:25.132356 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:25.132842 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:25.132801 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:26.131871 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:26.131840 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:26.132063 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:26.131983 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:27.131678 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:27.131630 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:27.132119 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:27.131629 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:27.132119 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:27.131795 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:27.132119 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:27.131916 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:28.131991 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:28.131957 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:28.132434 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:28.132098 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:29.131544 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:29.131509 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:29.131753 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:29.131639 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:29.131753 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:29.131669 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:29.131753 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:29.131741 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:30.131576 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:30.131535 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:30.132084 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:30.131653 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:31.131715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:31.131678 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:31.131715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:31.131699 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:31.132232 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.131803 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:31.132232 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.131915 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:31.693459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:31.693425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:31.693636 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.693575 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:31.693679 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.693639 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:47.693621822 +0000 UTC m=+34.159749372 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:31.793984 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:31.793918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:31.794120 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.794030 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:31.794120 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.794090 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:47.794074275 +0000 UTC m=+34.260201810 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:31.894763 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:31.894712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:31.894927 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.894896 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:31.894927 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.894919 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:31.895014 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.894930 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:31.895014 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:31.894989 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:47.894968579 +0000 UTC m=+34.361096129 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:32.132345 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:32.132259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:32.132782 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:32.132391 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:33.131793 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:33.131756 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:33.131793 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:33.131779 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:33.132047 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:33.131919 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:33.132047 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:33.132023 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:34.132953 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:34.132914 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:34.133358 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:34.133025 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:35.132024 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:35.131989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:35.132207 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:35.132005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:35.132207 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:35.132118 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:35.132207 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:35.132163 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:36.131229 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.131207 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:36.131511 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:36.131343 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:36.222415 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.222162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r5cbh" event={"ID":"9cfe8616-0e6e-43a0-86f3-74dbdaf03607","Type":"ContainerStarted","Data":"bc3117b83161a30a788427b0306f200374f3e4e64a25b121f959bdc2d58c46a8"} Jul 14 13:27:36.224009 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.223986 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" event={"ID":"cb94a58b8790d4f4e68607b8aa46901b","Type":"ContainerStarted","Data":"d12e50a86d6326315e517aa44c66ad6ed8cafeb2615bdcf4dda07f6268930008"} Jul 14 13:27:36.225333 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.225313 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerStarted","Data":"28879d74ac4f314eb7cfeffb40bd3dacc2c42dec6509f0445afb9ac0c2707006"} Jul 14 13:27:36.226577 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.226557 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t8wfc" event={"ID":"b4df8cf1-1b78-4cb1-a6b3-d0239d090817","Type":"ContainerStarted","Data":"f4204d2c4a4de7a7aa1a2f7f5bd1a59d82f337af453101ed0aa3ce87dc72c69b"} Jul 14 13:27:36.243675 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.243637 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r5cbh" podStartSLOduration=2.5844252770000002 podStartE2EDuration="22.243625855s" totalImagesPullingTime="19.659200578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:36.243243215 +0000 UTC m=+22.709370768" watchObservedRunningTime="2026-07-14 13:27:36.243625855 +0000 UTC m=+22.709753410" Jul 14 13:27:36.266344 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.266296 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-114.ec2.internal" podStartSLOduration=20.266285138 podStartE2EDuration="20.266285138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:36.266024348 +0000 UTC m=+22.732151901" watchObservedRunningTime="2026-07-14 13:27:36.266285138 +0000 UTC m=+22.732412693" Jul 14 13:27:36.327186 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:36.327074 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t8wfc" podStartSLOduration=1.6402838530000001 podStartE2EDuration="21.327055454s" totalImagesPullingTime="19.686771601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:36.285856714 +0000 UTC m=+22.751984270" watchObservedRunningTime="2026-07-14 13:27:36.327055454 +0000 UTC m=+22.793183009" Jul 14 13:27:37.132195 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.131984 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:37.133048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.131995 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:37.133048 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:37.132298 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:37.133048 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:37.132339 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:37.229531 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.229496 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62bn9" event={"ID":"3785a337-66d1-4949-8d42-deb5d4008646","Type":"ContainerStarted","Data":"63c4e81a9245fdb2501be6cc8d63015dbfab22868d5031f90e866fb878991f86"} Jul 14 13:27:37.231002 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.230977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkdmr" event={"ID":"0fefb8f4-1dd4-4d15-aa77-bb897fa3528c","Type":"ContainerStarted","Data":"036eff52f0dc9e7b2441aba942ad62711800a9ecbcebbc7d065d2bd9ffc2d29a"} Jul 14 13:27:37.232383 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.232362 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="28879d74ac4f314eb7cfeffb40bd3dacc2c42dec6509f0445afb9ac0c2707006" exitCode=0 Jul 14 13:27:37.232451 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.232430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"28879d74ac4f314eb7cfeffb40bd3dacc2c42dec6509f0445afb9ac0c2707006"} Jul 14 13:27:37.233874 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.233858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" event={"ID":"389763af-1d4f-4f88-8956-0c53b42c720f","Type":"ContainerStarted","Data":"0ad9f328f151e8b1cc7d74bd744f73a7373f536519ae5ce191c791c91f95aa34"} Jul 14 13:27:37.235166 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.235148 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdwtm" event={"ID":"b30953bd-322a-4251-b5c4-1fb7799a8384","Type":"ContainerStarted","Data":"e4a99ac50da9cedf14cfa1c63b611913cf7534273bcd264c526e6b10418024f4"} Jul 14 13:27:37.238933 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.238915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:27:37.239311 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239290 2568 generic.go:356] "Generic (PLEG): container finished" podID="745ddaf3-baf3-4b8a-b12b-cea1b04446e7" containerID="7c898902e716c9873cdc299f1051591a9b670700b030ebac3551d444db3b310d" exitCode=1 Jul 14 13:27:37.239398 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"4705092a75e3a6c1c3c7b3732c7c4ef09c3186b29bf32f1215f5c3f1aac729de"} Jul 14 13:27:37.239473 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"58e369136f66ffc56f6ac72cedad6112766f96c2fecb1ef582c117e3f78caa21"} Jul 14 13:27:37.239473 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239464 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"df6dc01171c09845210e8555ad7198accb6cbde92f6e56113666cd0a2cc82a0e"} Jul 14 13:27:37.239547 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"0afffd947a9165236f0b56bc788fe528928e7509702503bd5b47e20722c286ec"} Jul 14 13:27:37.239547 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerDied","Data":"7c898902e716c9873cdc299f1051591a9b670700b030ebac3551d444db3b310d"} Jul 14 13:27:37.239618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.239562 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"56db11bdde1a1ee546287deaf526345d2639bc896353d373ebe21bc2f4ccc0c9"} Jul 14 13:27:37.259204 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.259126 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-62bn9" podStartSLOduration=2.560569117 podStartE2EDuration="22.259112835s" totalImagesPullingTime="19.698543718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:37.256543405 +0000 UTC m=+23.722670959" watchObservedRunningTime="2026-07-14 13:27:37.259112835 +0000 UTC m=+23.725240388" Jul 14 13:27:37.278336 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.278292 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gdwtm" podStartSLOduration=2.5698506979999998 podStartE2EDuration="22.278277516s" totalImagesPullingTime="19.708426818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:37.278016016 +0000 UTC m=+23.744143570" watchObservedRunningTime="2026-07-14 13:27:37.278277516 +0000 UTC m=+23.744405069" Jul 14 13:27:37.330061 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.330001 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vkdmr" podStartSLOduration=3.547485696 podStartE2EDuration="23.329986537s" totalImagesPullingTime="19.782500841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:37.329469658 +0000 UTC m=+23.795597228" watchObservedRunningTime="2026-07-14 13:27:37.329986537 +0000 UTC m=+23.796114092" Jul 14 13:27:37.908466 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:37.908441 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 13:27:38.017466 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.017365 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T13:27:37.90846036Z","UUID":"36edc012-841e-402f-b5b1-3704c12e5ddd","Handler":null,"Name":"","Endpoint":""} Jul 14 13:27:38.019055 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.019035 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 13:27:38.019183 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.019071 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 13:27:38.131784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.131755 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:38.131975 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:38.131874 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:38.186394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.186363 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:38.187018 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.187004 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:38.220424 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.220397 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:38.220873 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.220860 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r5cbh" Jul 14 13:27:38.242992 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.242961 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" event={"ID":"389763af-1d4f-4f88-8956-0c53b42c720f","Type":"ContainerStarted","Data":"845685480badf34e563c35fe92d6bdd26593d6e331a4006d60509a052ca4eb5e"} Jul 14 13:27:38.244249 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:38.244225 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zpsd5" event={"ID":"121c8fbe-b4dd-42ff-8dba-798ef736dfc2","Type":"ContainerStarted","Data":"e274f68f7e09c6d5def3e186f60f3b45b463c86d3d99f42677e26deac037e1ac"} Jul 14 13:27:39.132072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:39.132044 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:39.132271 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:39.132167 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:39.132271 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:39.132197 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:39.132271 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:39.132259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:40.132553 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.132339 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:40.133119 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:40.132685 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:40.251280 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.251245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" event={"ID":"389763af-1d4f-4f88-8956-0c53b42c720f","Type":"ContainerStarted","Data":"96253dcb42c2d55bcd18b518a4e4939ee454fbcaf1cf02e60d00dd6f828427f7"} Jul 14 13:27:40.254938 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.254915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:27:40.255359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.255334 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"6253b7eb19eb21bc2b8b5571e67a7beef8b1f1dead93de590878ed92fa0f2fa8"} Jul 14 13:27:40.274743 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.274685 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k6l7p" podStartSLOduration=2.850923497 podStartE2EDuration="25.274671613s" totalImagesPullingTime="22.423748116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:40.274247983 +0000 UTC m=+26.740375538" watchObservedRunningTime="2026-07-14 13:27:40.274671613 +0000 UTC m=+26.740799166" Jul 14 13:27:40.275010 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:40.274984 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zpsd5" podStartSLOduration=5.577392281 podStartE2EDuration="25.274975341s" totalImagesPullingTime="19.69758306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:38.266593923 +0000 UTC m=+24.732721477" watchObservedRunningTime="2026-07-14 13:27:40.274975341 +0000 UTC m=+26.741102895" Jul 14 13:27:41.131498 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:41.131466 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:41.131682 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:41.131587 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:41.131682 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:41.131642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:41.131815 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:41.131770 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:42.131253 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.131221 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:42.131859 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:42.131363 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:42.262710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.262572 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:27:42.263018 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.262997 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"fdbd5779ceadb5d06656ee7f552afab1c616580b015aa7e3f2a222f6a3045ff6"} Jul 14 13:27:42.263299 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.263286 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:42.263495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.263480 2568 scope.go:117] "RemoveContainer" containerID="7c898902e716c9873cdc299f1051591a9b670700b030ebac3551d444db3b310d" Jul 14 13:27:42.278085 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:42.278064 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:43.131357 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.131319 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:43.131774 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.131325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:43.131774 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:43.131422 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:43.131774 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:43.131527 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:43.268190 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.268165 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:27:43.268544 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.268514 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" event={"ID":"745ddaf3-baf3-4b8a-b12b-cea1b04446e7","Type":"ContainerStarted","Data":"b6780c69695d9b378979cc845712385be3a2c6adbe6e8eff5e97194acc533e36"} Jul 14 13:27:43.268835 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.268806 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:43.268966 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.268872 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:43.270283 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.270256 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="a568a46eaf0d82baa1a0d7a3c676164ba8b023076fdc3719a5fe773ede128c06" exitCode=0 Jul 14 13:27:43.270383 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.270292 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"a568a46eaf0d82baa1a0d7a3c676164ba8b023076fdc3719a5fe773ede128c06"} Jul 14 13:27:43.284216 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.284196 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:27:43.332317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:43.332212 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" podStartSLOduration=9.224865026 podStartE2EDuration="29.332194574s" totalImagesPullingTime="20.107329548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:43.306575695 +0000 UTC m=+29.772703250" watchObservedRunningTime="2026-07-14 13:27:43.332194574 +0000 UTC m=+29.798322127" Jul 14 13:27:44.132711 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.132679 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:44.133102 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:44.132783 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:44.674208 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.674173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krb8t"] Jul 14 13:27:44.674377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.674299 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:44.674450 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:44.674404 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:44.677309 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.677282 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sdbxt"] Jul 14 13:27:44.677412 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.677376 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:44.677465 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:44.677450 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:44.679587 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.679561 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gzfsj"] Jul 14 13:27:44.679674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:44.679647 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:44.679753 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:44.679719 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:45.280342 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:45.280301 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="296127e97a0e892591a49382cb50435f4bf0d170a77ab63755028ae9ddfcb4e7" exitCode=0 Jul 14 13:27:45.280837 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:45.280393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"296127e97a0e892591a49382cb50435f4bf0d170a77ab63755028ae9ddfcb4e7"} Jul 14 13:27:46.132362 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:46.132165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:46.132524 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:46.132165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:46.132524 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:46.132454 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:46.132644 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:46.132550 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:46.132644 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:46.132177 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:46.132753 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:46.132667 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:47.287769 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:47.287721 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="c37ac199120e48f1e859c786b541e340c9b9638a7a5f61d28be67773b318cbf9" exitCode=0 Jul 14 13:27:47.287769 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:47.287777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"c37ac199120e48f1e859c786b541e340c9b9638a7a5f61d28be67773b318cbf9"} Jul 14 13:27:47.713438 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:47.713363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:47.713595 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.713506 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:47.713595 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.713572 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs podName:137eec1b-b24f-4c5d-9eb7-81601d40c532 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:19.713555288 +0000 UTC m=+66.179682821 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs") pod "network-metrics-daemon-sdbxt" (UID: "137eec1b-b24f-4c5d-9eb7-81601d40c532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:27:47.814534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:47.814496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:47.814719 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.814653 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:47.814799 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.814720 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret podName:d139e885-1141-43fc-a857-62e336ce02c5 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:19.814704427 +0000 UTC m=+66.280831964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret") pod "global-pull-secret-syncer-krb8t" (UID: "d139e885-1141-43fc-a857-62e336ce02c5") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:27:47.915281 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:47.915235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:47.915471 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.915446 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:27:47.915540 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.915475 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:27:47.915540 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.915487 2568 projected.go:196] Error preparing data for projected volume kube-api-access-b2j2c for pod openshift-network-diagnostics/network-check-target-gzfsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:47.915646 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:47.915552 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c podName:2ba452ed-4c7e-40ac-bbcf-da5e32db4f92 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:19.915529864 +0000 UTC m=+66.381657422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b2j2c" (UniqueName: "kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c") pod "network-check-target-gzfsj" (UID: "2ba452ed-4c7e-40ac-bbcf-da5e32db4f92") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:27:48.132108 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.132081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:48.132286 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.132081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:48.132286 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:48.132210 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sdbxt" podUID="137eec1b-b24f-4c5d-9eb7-81601d40c532" Jul 14 13:27:48.132286 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.132081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:48.132429 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:48.132297 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-krb8t" podUID="d139e885-1141-43fc-a857-62e336ce02c5" Jul 14 13:27:48.132429 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:48.132370 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gzfsj" podUID="2ba452ed-4c7e-40ac-bbcf-da5e32db4f92" Jul 14 13:27:48.885947 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.885860 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-114.ec2.internal" event="NodeReady" Jul 14 13:27:48.886318 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.886021 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 13:27:48.925262 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.925228 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx"] Jul 14 13:27:48.927113 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.927092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:48.930267 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.929046 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:27:48.931866 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.931835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 14 13:27:48.932282 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.931841 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.932820 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.932561 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-szll9\"" Jul 14 13:27:48.932965 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.932946 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.933139 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.933120 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ml48v"] Jul 14 13:27:48.933789 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.933770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:48.936114 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.936081 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 14 13:27:48.937215 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.937197 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c7b478588-9cfpm"] Jul 14 13:27:48.937329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.937310 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:48.938806 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.938786 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8"] Jul 14 13:27:48.938977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.938948 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:48.940009 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.939895 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 13:27:48.940126 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.940109 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jzn49\"" Jul 14 13:27:48.940479 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.940376 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 13:27:48.940540 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.940507 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 13:27:48.940775 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.940759 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b"] Jul 14 13:27:48.941231 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.940832 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" Jul 14 13:27:48.942675 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.942654 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.942990 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj"] Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943198 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943224 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hr4wp\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943399 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943436 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943660 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943666 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.943911 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.944057 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 13:27:48.944252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.942504 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.944756 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.944313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hh6kq\"" Jul 14 13:27:48.944995 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.944979 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 13:27:48.945386 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.945351 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-nrmmv"] Jul 14 13:27:48.945549 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.945518 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.945710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.945688 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.945942 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.945924 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-crk52\"" Jul 14 13:27:48.946254 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.946237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:48.947347 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.947327 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.949483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.949460 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28"] Jul 14 13:27:48.951529 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.951510 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.952358 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952336 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:48.952595 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.951520 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 14 13:27:48.952823 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.951670 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 14 13:27:48.953112 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.951777 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.953276 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952065 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qddkt\"" Jul 14 13:27:48.953476 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952144 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.953476 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952196 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dgb5n\"" Jul 14 13:27:48.953476 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952216 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 14 13:27:48.956679 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.956198 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd"] Jul 14 13:27:48.962146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.952284 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 13:27:48.962146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.957953 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 14 13:27:48.962146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.961416 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.962146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.961551 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:48.962723 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.951574 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.963376 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.963359 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w5652\"" Jul 14 13:27:48.963819 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.963687 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g9z48"] Jul 14 13:27:48.966590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.965489 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 14 13:27:48.966590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.965820 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.966590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.966238 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 13:27:48.966590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.966377 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.968745 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.966894 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" Jul 14 13:27:48.968745 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.968117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 13:27:48.968745 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.968257 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wqwgc\"" Jul 14 13:27:48.969130 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.969114 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ls9n8"] Jul 14 13:27:48.969766 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.969749 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.970253 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.969371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:48.971627 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.971604 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.971865 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.971840 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.972454 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.972267 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-c8djg\"" Jul 14 13:27:48.973173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973154 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973186 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973200 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973214 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ml48v"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973228 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973246 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b"] Jul 14 13:27:48.973273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973269 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd"] Jul 14 13:27:48.973558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973283 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c7b478588-9cfpm"] Jul 14 13:27:48.973558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973295 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8"] Jul 14 13:27:48.973558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973378 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:48.973558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.973392 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 13:27:48.974252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.974232 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 14 13:27:48.975148 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.975127 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-755dc\"" Jul 14 13:27:48.976049 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976028 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 13:27:48.976049 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976046 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 13:27:48.976176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976109 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 13:27:48.976176 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976038 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 13:27:48.976533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976496 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rczr\"" Jul 14 13:27:48.976925 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.976840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 13:27:48.978267 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.978241 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ls9n8"] Jul 14 13:27:48.984265 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.984163 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-nrmmv"] Jul 14 13:27:48.985913 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:48.985893 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9z48"] Jul 14 13:27:49.023151 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023294 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.023294 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023192 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023294 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023294 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-snapshots\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt7kt\" (UniqueName: \"kubernetes.io/projected/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-kube-api-access-zt7kt\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.023497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38ec00a8-80ff-4fde-a998-e065e0169859-serving-cert\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s78mx\" (UniqueName: \"kubernetes.io/projected/38ec00a8-80ff-4fde-a998-e065e0169859-kube-api-access-s78mx\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-tmp\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq977\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.023677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023654 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023890 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.023890 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023890 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.023890 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.023805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.124362 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-snapshots\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrl4\" (UniqueName: \"kubernetes.io/projected/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-kube-api-access-zsrl4\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq977\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.124530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a7b5ed4-167d-4a0c-945b-b70e878a8026-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a009c8-8970-4d30-b26e-44c8f404767b-serving-cert\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86639ef1-7e07-495c-a375-1d8a0bf140cf-config\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.124880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.124875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.125229 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.125287 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tl62\" (UniqueName: \"kubernetes.io/projected/86639ef1-7e07-495c-a375-1d8a0bf140cf-kube-api-access-4tl62\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.125469 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125204 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-snapshots\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b314fe-62fc-4a4c-9fce-c782da4854a4-tmp-dir\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.125712 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.125753 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.125794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.125841 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.625811986 +0000 UTC m=+36.091939544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-stats-auth\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126116 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.126181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126152 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mr8f\" (UniqueName: \"kubernetes.io/projected/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-kube-api-access-6mr8f\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjpm\" (UniqueName: \"kubernetes.io/projected/b4288846-6ba6-4eac-adf3-2d9136a129cb-kube-api-access-ptjpm\") pod \"network-check-source-869dcfcdcd-2fmrd\" (UID: \"b4288846-6ba6-4eac-adf3-2d9136a129cb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt7kt\" (UniqueName: \"kubernetes.io/projected/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-kube-api-access-zt7kt\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38ec00a8-80ff-4fde-a998-e065e0169859-serving-cert\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s78mx\" (UniqueName: \"kubernetes.io/projected/38ec00a8-80ff-4fde-a998-e065e0169859-kube-api-access-s78mx\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkkbb\" (UniqueName: \"kubernetes.io/projected/f751502f-83f6-45a5-9847-c556e5aaacd1-kube-api-access-qkkbb\") pod \"volume-data-source-validator-78495597f6-btsv8\" (UID: \"f751502f-83f6-45a5-9847-c556e5aaacd1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-cert\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-tmp\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126432 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b314fe-62fc-4a4c-9fce-c782da4854a4-config-volume\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86639ef1-7e07-495c-a375-1d8a0bf140cf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a7b5ed4-167d-4a0c-945b-b70e878a8026-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126524 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttf5\" (UniqueName: \"kubernetes.io/projected/7a7b5ed4-167d-4a0c-945b-b70e878a8026-kube-api-access-lttf5\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126575 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-config\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.126652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126638 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf2rn\" (UniqueName: \"kubernetes.io/projected/94a009c8-8970-4d30-b26e-44c8f404767b-kube-api-access-wf2rn\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126679 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-default-certificate\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-trusted-ca\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.126926 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38ec00a8-80ff-4fde-a998-e065e0169859-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127009 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfd4l\" (UniqueName: \"kubernetes.io/projected/f9e29ed3-0274-43f7-8d29-848cec6de6a9-kube-api-access-nfd4l\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj9gs\" (UniqueName: \"kubernetes.io/projected/40b314fe-62fc-4a4c-9fce-c782da4854a4-kube-api-access-xj9gs\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.127194 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38ec00a8-80ff-4fde-a998-e065e0169859-tmp\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.127317 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.127239 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.627224502 +0000 UTC m=+36.093352067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:49.127880 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127422 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.127944 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.127905 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.131323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.131291 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.131323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.131306 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.131450 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.131324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38ec00a8-80ff-4fde-a998-e065e0169859-serving-cert\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.139639 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.139560 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.142023 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.141995 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s78mx\" (UniqueName: \"kubernetes.io/projected/38ec00a8-80ff-4fde-a998-e065e0169859-kube-api-access-s78mx\") pod \"insights-operator-68d9dd94cf-ml48v\" (UID: \"38ec00a8-80ff-4fde-a998-e065e0169859\") " pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.142145 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.142121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt7kt\" (UniqueName: \"kubernetes.io/projected/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-kube-api-access-zt7kt\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.143522 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.143504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq977\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.228106 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tl62\" (UniqueName: \"kubernetes.io/projected/86639ef1-7e07-495c-a375-1d8a0bf140cf-kube-api-access-4tl62\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.228106 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b314fe-62fc-4a4c-9fce-c782da4854a4-tmp-dir\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-stats-auth\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228176 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mr8f\" (UniqueName: \"kubernetes.io/projected/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-kube-api-access-6mr8f\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228203 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjpm\" (UniqueName: \"kubernetes.io/projected/b4288846-6ba6-4eac-adf3-2d9136a129cb-kube-api-access-ptjpm\") pod \"network-check-source-869dcfcdcd-2fmrd\" (UID: \"b4288846-6ba6-4eac-adf3-2d9136a129cb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkkbb\" (UniqueName: \"kubernetes.io/projected/f751502f-83f6-45a5-9847-c556e5aaacd1-kube-api-access-qkkbb\") pod \"volume-data-source-validator-78495597f6-btsv8\" (UID: \"f751502f-83f6-45a5-9847-c556e5aaacd1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-cert\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.228351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b314fe-62fc-4a4c-9fce-c782da4854a4-config-volume\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86639ef1-7e07-495c-a375-1d8a0bf140cf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228481 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a7b5ed4-167d-4a0c-945b-b70e878a8026-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228500 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40b314fe-62fc-4a4c-9fce-c782da4854a4-tmp-dir\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttf5\" (UniqueName: \"kubernetes.io/projected/7a7b5ed4-167d-4a0c-945b-b70e878a8026-kube-api-access-lttf5\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-config\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.228626 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf2rn\" (UniqueName: \"kubernetes.io/projected/94a009c8-8970-4d30-b26e-44c8f404767b-kube-api-access-wf2rn\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-default-certificate\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228661 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-trusted-ca\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228702 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228751 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfd4l\" (UniqueName: \"kubernetes.io/projected/f9e29ed3-0274-43f7-8d29-848cec6de6a9-kube-api-access-nfd4l\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228798 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj9gs\" (UniqueName: \"kubernetes.io/projected/40b314fe-62fc-4a4c-9fce-c782da4854a4-kube-api-access-xj9gs\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrl4\" (UniqueName: \"kubernetes.io/projected/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-kube-api-access-zsrl4\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a7b5ed4-167d-4a0c-945b-b70e878a8026-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.228936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.229359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.229359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.229005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a009c8-8970-4d30-b26e-44c8f404767b-serving-cert\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.229359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.229029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86639ef1-7e07-495c-a375-1d8a0bf140cf-config\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.229536 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.229512 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-config\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.229610 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.229589 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86639ef1-7e07-495c-a375-1d8a0bf140cf-config\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.229610 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.229601 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/94a009c8-8970-4d30-b26e-44c8f404767b-trusted-ca\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.229704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229618 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:27:49.229704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229679 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.729662294 +0000 UTC m=+36.195789853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:27:49.229704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229688 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:49.229927 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.228904 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40b314fe-62fc-4a4c-9fce-c782da4854a4-config-volume\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.229927 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229765 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.729747933 +0000 UTC m=+36.195875464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:27:49.229927 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229846 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.729834584 +0000 UTC m=+36.195962121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:27:49.229927 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229899 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:27:49.230238 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.229932 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:49.729922312 +0000 UTC m=+36.196049881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:27:49.230238 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.230023 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a7b5ed4-167d-4a0c-945b-b70e878a8026-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.231707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.231663 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-cert\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.231976 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.231885 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86639ef1-7e07-495c-a375-1d8a0bf140cf-serving-cert\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.232171 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.232125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a7b5ed4-167d-4a0c-945b-b70e878a8026-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.232802 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.232578 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/94a009c8-8970-4d30-b26e-44c8f404767b-serving-cert\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.232893 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.232824 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-default-certificate\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.233706 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.233683 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-stats-auth\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.242868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.242119 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfd4l\" (UniqueName: \"kubernetes.io/projected/f9e29ed3-0274-43f7-8d29-848cec6de6a9-kube-api-access-nfd4l\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.242868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.242230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkkbb\" (UniqueName: \"kubernetes.io/projected/f751502f-83f6-45a5-9847-c556e5aaacd1-kube-api-access-qkkbb\") pod \"volume-data-source-validator-78495597f6-btsv8\" (UID: \"f751502f-83f6-45a5-9847-c556e5aaacd1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" Jul 14 13:27:49.242868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.242516 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tl62\" (UniqueName: \"kubernetes.io/projected/86639ef1-7e07-495c-a375-1d8a0bf140cf-kube-api-access-4tl62\") pod \"service-ca-operator-7695fc6c8c-94b28\" (UID: \"86639ef1-7e07-495c-a375-1d8a0bf140cf\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.242868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.242632 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttf5\" (UniqueName: \"kubernetes.io/projected/7a7b5ed4-167d-4a0c-945b-b70e878a8026-kube-api-access-lttf5\") pod \"kube-storage-version-migrator-operator-799cb7f958-wdvzj\" (UID: \"7a7b5ed4-167d-4a0c-945b-b70e878a8026\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.242868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.242865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj9gs\" (UniqueName: \"kubernetes.io/projected/40b314fe-62fc-4a4c-9fce-c782da4854a4-kube-api-access-xj9gs\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.244415 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.244376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf2rn\" (UniqueName: \"kubernetes.io/projected/94a009c8-8970-4d30-b26e-44c8f404767b-kube-api-access-wf2rn\") pod \"console-operator-7866497494-nrmmv\" (UID: \"94a009c8-8970-4d30-b26e-44c8f404767b\") " pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.244971 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.244926 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrl4\" (UniqueName: \"kubernetes.io/projected/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-kube-api-access-zsrl4\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.245054 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.245030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mr8f\" (UniqueName: \"kubernetes.io/projected/d4cdbd15-22b6-4b58-8ddc-1f93d1323014-kube-api-access-6mr8f\") pod \"ingress-canary-g9z48\" (UID: \"d4cdbd15-22b6-4b58-8ddc-1f93d1323014\") " pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.245254 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.245211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjpm\" (UniqueName: \"kubernetes.io/projected/b4288846-6ba6-4eac-adf3-2d9136a129cb-kube-api-access-ptjpm\") pod \"network-check-source-869dcfcdcd-2fmrd\" (UID: \"b4288846-6ba6-4eac-adf3-2d9136a129cb\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" Jul 14 13:27:49.270954 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.270927 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" Jul 14 13:27:49.289310 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.289280 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" Jul 14 13:27:49.302819 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.302794 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" Jul 14 13:27:49.310660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.310637 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:27:49.317332 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.317292 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" Jul 14 13:27:49.326092 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.325827 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" Jul 14 13:27:49.334029 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.333843 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9z48" Jul 14 13:27:49.507876 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.505586 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-ml48v"] Jul 14 13:27:49.552529 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.552439 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8"] Jul 14 13:27:49.579783 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.579633 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj"] Jul 14 13:27:49.583940 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:49.583905 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a7b5ed4_167d_4a0c_945b_b70e878a8026.slice/crio-a10397786a366ff138f52261b254f1ec6a78e66423951482af6f2d28a2bc82a3 WatchSource:0}: Error finding container a10397786a366ff138f52261b254f1ec6a78e66423951482af6f2d28a2bc82a3: Status 404 returned error can't find the container with id a10397786a366ff138f52261b254f1ec6a78e66423951482af6f2d28a2bc82a3 Jul 14 13:27:49.599676 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.599644 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28"] Jul 14 13:27:49.602985 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:49.602950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86639ef1_7e07_495c_a375_1d8a0bf140cf.slice/crio-ed7a55425867356e38f11c30526eac22c9c1c98e3091c1a1ff25e11ceaa93274 WatchSource:0}: Error finding container ed7a55425867356e38f11c30526eac22c9c1c98e3091c1a1ff25e11ceaa93274: Status 404 returned error can't find the container with id ed7a55425867356e38f11c30526eac22c9c1c98e3091c1a1ff25e11ceaa93274 Jul 14 13:27:49.610489 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.610465 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-nrmmv"] Jul 14 13:27:49.613190 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.613055 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd"] Jul 14 13:27:49.614485 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:49.614458 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a009c8_8970_4d30_b26e_44c8f404767b.slice/crio-8bb7112f0fc98633d1ae7760c484b5ab40ed47f835d608c418b0047d3e4f5216 WatchSource:0}: Error finding container 8bb7112f0fc98633d1ae7760c484b5ab40ed47f835d608c418b0047d3e4f5216: Status 404 returned error can't find the container with id 8bb7112f0fc98633d1ae7760c484b5ab40ed47f835d608c418b0047d3e4f5216 Jul 14 13:27:49.617072 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:49.617043 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4288846_6ba6_4eac_adf3_2d9136a129cb.slice/crio-e03c52dd79c5c6f9ade44f3c263dbb5977ad171aafb3cc2565060bb85dd286f1 WatchSource:0}: Error finding container e03c52dd79c5c6f9ade44f3c263dbb5977ad171aafb3cc2565060bb85dd286f1: Status 404 returned error can't find the container with id e03c52dd79c5c6f9ade44f3c263dbb5977ad171aafb3cc2565060bb85dd286f1 Jul 14 13:27:49.627337 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.627307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9z48"] Jul 14 13:27:49.630319 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:27:49.630292 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4cdbd15_22b6_4b58_8ddc_1f93d1323014.slice/crio-134502fa7fd714820d3587e94e5c030f35f3528dede9aa255925f5551715715d WatchSource:0}: Error finding container 134502fa7fd714820d3587e94e5c030f35f3528dede9aa255925f5551715715d: Status 404 returned error can't find the container with id 134502fa7fd714820d3587e94e5c030f35f3528dede9aa255925f5551715715d Jul 14 13:27:49.633025 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.633001 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:49.633168 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.633148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:49.633286 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.633157 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:27:49.633330 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.633300 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:27:49.633380 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.633212 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:49.633414 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.633363 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.633345516 +0000 UTC m=+37.099473060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:27:49.633458 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.633436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.633416193 +0000 UTC m=+37.099543741 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:49.734439 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.734348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.734439 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.734409 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:49.734439 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.734434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:49.734450 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734510 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734536 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734579 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.734559442 +0000 UTC m=+37.200686988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734587 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734611 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.734588849 +0000 UTC m=+37.200716386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734626 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.734617718 +0000 UTC m=+37.200745254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:27:49.734713 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:49.734640 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:50.734632981 +0000 UTC m=+37.200760517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:27:50.131773 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.131719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:27:50.132339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.131719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:27:50.132339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.131719 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:27:50.134911 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.134865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:27:50.134911 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.134879 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:27:50.136281 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.136140 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7sfg\"" Jul 14 13:27:50.136281 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.136151 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rgn8m\"" Jul 14 13:27:50.299998 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.299956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9z48" event={"ID":"d4cdbd15-22b6-4b58-8ddc-1f93d1323014","Type":"ContainerStarted","Data":"134502fa7fd714820d3587e94e5c030f35f3528dede9aa255925f5551715715d"} Jul 14 13:27:50.301224 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.301174 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" event={"ID":"38ec00a8-80ff-4fde-a998-e065e0169859","Type":"ContainerStarted","Data":"af14de6a6cfa1fc6988d4cf31b7d025c010c5bbfcd22767db921d797ac987ae9"} Jul 14 13:27:50.302324 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.302297 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" event={"ID":"86639ef1-7e07-495c-a375-1d8a0bf140cf","Type":"ContainerStarted","Data":"ed7a55425867356e38f11c30526eac22c9c1c98e3091c1a1ff25e11ceaa93274"} Jul 14 13:27:50.303455 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.303434 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" event={"ID":"7a7b5ed4-167d-4a0c-945b-b70e878a8026","Type":"ContainerStarted","Data":"a10397786a366ff138f52261b254f1ec6a78e66423951482af6f2d28a2bc82a3"} Jul 14 13:27:50.304575 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.304553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" event={"ID":"b4288846-6ba6-4eac-adf3-2d9136a129cb","Type":"ContainerStarted","Data":"e03c52dd79c5c6f9ade44f3c263dbb5977ad171aafb3cc2565060bb85dd286f1"} Jul 14 13:27:50.305685 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.305663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nrmmv" event={"ID":"94a009c8-8970-4d30-b26e-44c8f404767b","Type":"ContainerStarted","Data":"8bb7112f0fc98633d1ae7760c484b5ab40ed47f835d608c418b0047d3e4f5216"} Jul 14 13:27:50.306686 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.306669 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" event={"ID":"f751502f-83f6-45a5-9847-c556e5aaacd1","Type":"ContainerStarted","Data":"8642c734f6131491a8231829245fb4a69d387d585cae81a4ca1928b82699b55f"} Jul 14 13:27:50.643525 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.643488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:50.643704 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.643583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:50.643704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.643636 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:27:50.643704 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.643660 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:27:50.643957 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.643750 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.643720268 +0000 UTC m=+39.109847804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:27:50.643957 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.643765 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:50.643957 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.643836 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.643817051 +0000 UTC m=+39.109944583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:50.744937 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.744899 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:50.745112 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.744960 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:50.745112 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.744996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:50.745112 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:50.745019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:50.745112 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745058 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745133 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.745113247 +0000 UTC m=+39.211240792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745149 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.745140915 +0000 UTC m=+39.211268447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745159 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745177 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745224 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.745204219 +0000 UTC m=+39.211331772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:27:50.745323 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:50.745244 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:52.745233983 +0000 UTC m=+39.211361515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:27:52.318449 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.318412 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="1ae41531ad19ba1b385f1072dfb981491ca92ff642fd66a2ce1197571f291ae7" exitCode=0 Jul 14 13:27:52.318909 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.318505 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"1ae41531ad19ba1b385f1072dfb981491ca92ff642fd66a2ce1197571f291ae7"} Jul 14 13:27:52.663784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.663358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:52.664008 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.663908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:52.664164 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.663721 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:27:52.664164 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.664137 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:27:52.664343 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.664194 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.664175298 +0000 UTC m=+43.130302833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:27:52.664343 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.664121 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:52.664343 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.664334 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.664323001 +0000 UTC m=+43.130450537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.765427 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.765480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.765614 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:52.765675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.765833 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.765896 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.765876777 +0000 UTC m=+43.232004311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.766181 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.766162849 +0000 UTC m=+43.232290388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.766247 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.766283 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.766271429 +0000 UTC m=+43.232398967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.766340 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:27:52.766421 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:52.766371 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:27:56.766360622 +0000 UTC m=+43.232488157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:27:53.331673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:53.330803 2568 generic.go:356] "Generic (PLEG): container finished" podID="581c0fd4-fdfb-44fd-992c-f8728532e277" containerID="10960e4effb36038ae150a1775317407eee6f0142f05aab50f8e43766bd124bf" exitCode=0 Jul 14 13:27:53.331673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:53.330853 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerDied","Data":"10960e4effb36038ae150a1775317407eee6f0142f05aab50f8e43766bd124bf"} Jul 14 13:27:56.705455 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.705406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.705537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.705574 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.705635 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.705648 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.705630313 +0000 UTC m=+51.171757845 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.705650 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:27:56.706104 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.705707 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.70569467 +0000 UTC m=+51.171822229 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:27:56.805933 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.805897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:27:56.805933 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.805938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:56.806171 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806047 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:27:56.806171 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.806054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:27:56.806171 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806125 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.806101763 +0000 UTC m=+51.272229317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:27:56.806171 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806128 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:27:56.806348 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806211 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.806186 +0000 UTC m=+51.272313544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:27:56.806348 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806253 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.806240282 +0000 UTC m=+51.272367818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:27:56.806448 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:56.806329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:27:56.806448 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806406 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:27:56.806448 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:27:56.806443 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.806433524 +0000 UTC m=+51.272561056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:27:59.349786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.349699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" event={"ID":"b4288846-6ba6-4eac-adf3-2d9136a129cb","Type":"ContainerStarted","Data":"c24b9103c120f690b1d9d4354653bc6432d59c91caf6e93f2d87119507e27084"} Jul 14 13:27:59.351792 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.351766 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/0.log" Jul 14 13:27:59.351926 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.351810 2568 generic.go:356] "Generic (PLEG): container finished" podID="94a009c8-8970-4d30-b26e-44c8f404767b" containerID="586a49750f07083ac954ab9f8f4b041b43488db668e0a3767834536e51faf64f" exitCode=255 Jul 14 13:27:59.351926 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.351895 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nrmmv" event={"ID":"94a009c8-8970-4d30-b26e-44c8f404767b","Type":"ContainerDied","Data":"586a49750f07083ac954ab9f8f4b041b43488db668e0a3767834536e51faf64f"} Jul 14 13:27:59.352433 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.352416 2568 scope.go:117] "RemoveContainer" containerID="586a49750f07083ac954ab9f8f4b041b43488db668e0a3767834536e51faf64f" Jul 14 13:27:59.353607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.353573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" event={"ID":"f751502f-83f6-45a5-9847-c556e5aaacd1","Type":"ContainerStarted","Data":"b208096c182afbcc7b3a929e7633ba03718ba93c1343476618c34c1c85893d24"} Jul 14 13:27:59.357958 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.357935 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nbgng" event={"ID":"581c0fd4-fdfb-44fd-992c-f8728532e277","Type":"ContainerStarted","Data":"02b1a47b4b6ce6e296ca585859c45a0cbb6cb94d2e8ca5fb2486dfd357bc066f"} Jul 14 13:27:59.359409 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.359386 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9z48" event={"ID":"d4cdbd15-22b6-4b58-8ddc-1f93d1323014","Type":"ContainerStarted","Data":"d7a046f32a58804dc2f6374c7cecb7f8ec87846762b4df5dde079ed1dd7fedb5"} Jul 14 13:27:59.360939 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.360916 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" event={"ID":"38ec00a8-80ff-4fde-a998-e065e0169859","Type":"ContainerStarted","Data":"eb3edd47381e810ad719651a9ea3ca1428ee1e88e486c12734fd71bd12650caa"} Jul 14 13:27:59.362375 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.362357 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" event={"ID":"86639ef1-7e07-495c-a375-1d8a0bf140cf","Type":"ContainerStarted","Data":"3eafa7b497c86565ebc9ffb58960e0edce4553b7c47361caff82283f5907fc49"} Jul 14 13:27:59.363884 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.363853 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" event={"ID":"7a7b5ed4-167d-4a0c-945b-b70e878a8026","Type":"ContainerStarted","Data":"728a096552c09ff7872b4f4bb24636271201381492fa5e141502315af85b8f9e"} Jul 14 13:27:59.378242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.378187 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-2fmrd" podStartSLOduration=4.152431395 podStartE2EDuration="13.378165964s" totalImagesPullingTime="9.225734569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.377434921 +0000 UTC m=+45.843562476" watchObservedRunningTime="2026-07-14 13:27:59.378165964 +0000 UTC m=+45.844293519" Jul 14 13:27:59.439752 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.439672 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g9z48" podStartSLOduration=2.229251481 podStartE2EDuration="11.439651567s" totalImagesPullingTime="9.210400086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.415644261 +0000 UTC m=+45.881771816" watchObservedRunningTime="2026-07-14 13:27:59.439651567 +0000 UTC m=+45.905779101" Jul 14 13:27:59.469159 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.469087 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nbgng" podStartSLOduration=14.499787407 podStartE2EDuration="45.469065496s" totalImagesPullingTime="30.969278089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.46747418 +0000 UTC m=+45.933601739" watchObservedRunningTime="2026-07-14 13:27:59.469065496 +0000 UTC m=+45.935193051" Jul 14 13:27:59.469361 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.469334 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" podStartSLOduration=15.148373688 podStartE2EDuration="24.469325409s" totalImagesPullingTime="9.320951721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.439940328 +0000 UTC m=+45.906067883" watchObservedRunningTime="2026-07-14 13:27:59.469325409 +0000 UTC m=+45.935452964" Jul 14 13:27:59.495905 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.495840 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-btsv8" podStartSLOduration=14.623369093 podStartE2EDuration="18.495819923s" totalImagesPullingTime="3.87245083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.495212393 +0000 UTC m=+45.961339948" watchObservedRunningTime="2026-07-14 13:27:59.495819923 +0000 UTC m=+45.961947479" Jul 14 13:27:59.522987 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:27:59.522920 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" podStartSLOduration=5.285237187 podStartE2EDuration="14.522895954s" totalImagesPullingTime="9.237658767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.521413962 +0000 UTC m=+45.987541517" watchObservedRunningTime="2026-07-14 13:27:59.522895954 +0000 UTC m=+45.989023502" Jul 14 13:28:00.370432 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.370406 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:28:00.370870 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.370853 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/0.log" Jul 14 13:28:00.370919 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.370888 2568 generic.go:356] "Generic (PLEG): container finished" podID="94a009c8-8970-4d30-b26e-44c8f404767b" containerID="5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8" exitCode=255 Jul 14 13:28:00.371027 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.371007 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nrmmv" event={"ID":"94a009c8-8970-4d30-b26e-44c8f404767b","Type":"ContainerDied","Data":"5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8"} Jul 14 13:28:00.371168 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.371073 2568 scope.go:117] "RemoveContainer" containerID="586a49750f07083ac954ab9f8f4b041b43488db668e0a3767834536e51faf64f" Jul 14 13:28:00.371236 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.371223 2568 scope.go:117] "RemoveContainer" containerID="5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8" Jul 14 13:28:00.371510 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:00.371482 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nrmmv_openshift-console-operator(94a009c8-8970-4d30-b26e-44c8f404767b)\"" pod="openshift-console-operator/console-operator-7866497494-nrmmv" podUID="94a009c8-8970-4d30-b26e-44c8f404767b" Jul 14 13:28:00.394714 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.394670 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" podStartSLOduration=5.131277795 podStartE2EDuration="14.394655668s" totalImagesPullingTime="9.263377873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:27:59.55716265 +0000 UTC m=+46.023290205" watchObservedRunningTime="2026-07-14 13:28:00.394655668 +0000 UTC m=+46.860783291" Jul 14 13:28:00.535406 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.535373 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lwqjt"] Jul 14 13:28:00.565417 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.565390 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lwqjt"] Jul 14 13:28:00.565591 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.565532 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.568465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.568441 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 13:28:00.568597 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.568559 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 13:28:00.568698 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.568679 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tbwpd\"" Jul 14 13:28:00.644040 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.643968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.644040 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.644005 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f731f2a-2763-4ea2-b8a3-bf991341b06c-data-volume\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.644040 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.644041 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbd88\" (UniqueName: \"kubernetes.io/projected/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-api-access-hbd88\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.644248 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.644075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.644248 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.644127 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f731f2a-2763-4ea2-b8a3-bf991341b06c-crio-socket\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f731f2a-2763-4ea2-b8a3-bf991341b06c-data-volume\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744659 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbd88\" (UniqueName: \"kubernetes.io/projected/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-api-access-hbd88\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:00.744698 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:00.744787 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:01.244768512 +0000 UTC m=+47.710896051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : configmap "kube-rbac-proxy" not found Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f731f2a-2763-4ea2-b8a3-bf991341b06c-crio-socket\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.744872 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:00.744861 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:00.745132 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:00.744922 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:01.244904213 +0000 UTC m=+47.711031753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:00.745132 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.744948 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8f731f2a-2763-4ea2-b8a3-bf991341b06c-crio-socket\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.745132 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.745031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8f731f2a-2763-4ea2-b8a3-bf991341b06c-data-volume\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:00.756826 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:00.756800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbd88\" (UniqueName: \"kubernetes.io/projected/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-api-access-hbd88\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:01.249668 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:01.249630 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:01.249858 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:01.249696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:01.249858 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:01.249804 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:01.249993 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:01.249870 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:01.249993 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:01.249886 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:02.249863837 +0000 UTC m=+48.715991376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : configmap "kube-rbac-proxy" not found Jul 14 13:28:01.249993 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:01.249921 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:02.249908134 +0000 UTC m=+48.716035666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:01.375938 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:01.375909 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:28:01.376305 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:01.376245 2568 scope.go:117] "RemoveContainer" containerID="5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8" Jul 14 13:28:01.376408 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:01.376389 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nrmmv_openshift-console-operator(94a009c8-8970-4d30-b26e-44c8f404767b)\"" pod="openshift-console-operator/console-operator-7866497494-nrmmv" podUID="94a009c8-8970-4d30-b26e-44c8f404767b" Jul 14 13:28:02.260185 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:02.260145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:02.260369 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:02.260206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:02.260369 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:02.260269 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:02.260369 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:02.260318 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:02.260369 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:02.260341 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.260322354 +0000 UTC m=+50.726449906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : configmap "kube-rbac-proxy" not found Jul 14 13:28:02.260369 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:02.260359 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:04.260348391 +0000 UTC m=+50.726475923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:03.134663 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:03.134637 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdwtm_b30953bd-322a-4251-b5c4-1fb7799a8384/dns-node-resolver/0.log" Jul 14 13:28:04.278294 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.278259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:04.278667 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.278318 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:04.278667 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.278375 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:04.278667 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.278438 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:08.278422069 +0000 UTC m=+54.744549602 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : configmap "kube-rbac-proxy" not found Jul 14 13:28:04.278667 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.278448 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:04.278667 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.278485 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:08.278474647 +0000 UTC m=+54.744602180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:04.520914 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.520885 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8wfc_b4df8cf1-1b78-4cb1-a6b3-d0239d090817/node-ca/0.log" Jul 14 13:28:04.782932 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.782904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:28:04.783061 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.783016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:04.783109 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.783052 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 14 13:28:04.783141 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.783123 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls podName:c31b1f5b-825c-4836-ba9f-ba304b7d4c45 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.783106373 +0000 UTC m=+67.249233908 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zj5sx" (UID: "c31b1f5b-825c-4836-ba9f-ba304b7d4c45") : secret "cluster-monitoring-operator-tls" not found Jul 14 13:28:04.783183 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.783140 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 13:28:04.783183 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.783152 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b8557cd44-8hwj2: secret "image-registry-tls" not found Jul 14 13:28:04.783243 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.783186 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls podName:eee0bc73-1865-422b-8b11-726b3d376674 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.783175584 +0000 UTC m=+67.249303150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls") pod "image-registry-6b8557cd44-8hwj2" (UID: "eee0bc73-1865-422b-8b11-726b3d376674") : secret "image-registry-tls" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.884831 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.884919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.884966 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:04.884998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.885244 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.885223011 +0000 UTC m=+67.351350563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : configmap references non-existent config key: service-ca.crt Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.885774 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.885835 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs podName:93b1c003-057f-48ca-abbb-e1f6b4b3bbc9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.885818437 +0000 UTC m=+67.351945968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs") pod "router-default-c7b478588-9cfpm" (UID: "93b1c003-057f-48ca-abbb-e1f6b4b3bbc9") : secret "router-metrics-certs-default" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.885910 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.885947 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls podName:f9e29ed3-0274-43f7-8d29-848cec6de6a9 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.885930282 +0000 UTC m=+67.352057827 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls") pod "cluster-samples-operator-65db858894-swx7b" (UID: "f9e29ed3-0274-43f7-8d29-848cec6de6a9") : secret "samples-operator-tls" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.886006 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 13:28:04.887655 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:04.886040 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls podName:40b314fe-62fc-4a4c-9fce-c782da4854a4 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:20.886030981 +0000 UTC m=+67.352158513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls") pod "dns-default-ls9n8" (UID: "40b314fe-62fc-4a4c-9fce-c782da4854a4") : secret "dns-default-metrics-tls" not found Jul 14 13:28:05.123007 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:05.122937 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g9z48_d4cdbd15-22b6-4b58-8ddc-1f93d1323014/serve-healthcheck-canary/0.log" Jul 14 13:28:05.922171 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:05.922141 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-wdvzj_7a7b5ed4-167d-4a0c-945b-b70e878a8026/kube-storage-version-migrator-operator/0.log" Jul 14 13:28:08.316600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:08.316564 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:08.317110 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:08.316637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:08.317110 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:08.316704 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 13:28:08.317110 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:08.316780 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:08.317110 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:08.316799 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:16.316782347 +0000 UTC m=+62.782909878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : configmap "kube-rbac-proxy" not found Jul 14 13:28:08.317110 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:08.316831 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:16.316820012 +0000 UTC m=+62.782947544 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:09.311465 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:09.311430 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:28:09.311627 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:09.311496 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:28:09.311872 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:09.311852 2568 scope.go:117] "RemoveContainer" containerID="5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8" Jul 14 13:28:09.312031 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:09.312012 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-nrmmv_openshift-console-operator(94a009c8-8970-4d30-b26e-44c8f404767b)\"" pod="openshift-console-operator/console-operator-7866497494-nrmmv" podUID="94a009c8-8970-4d30-b26e-44c8f404767b" Jul 14 13:28:15.296562 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:15.296532 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b6mvv" Jul 14 13:28:16.389152 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:16.389099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:16.389541 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:16.389182 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:16.389541 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:16.389289 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 13:28:16.389541 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:16.389342 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls podName:8f731f2a-2763-4ea2-b8a3-bf991341b06c nodeName:}" failed. No retries permitted until 2026-07-14 13:28:32.389327582 +0000 UTC m=+78.855455113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lwqjt" (UID: "8f731f2a-2763-4ea2-b8a3-bf991341b06c") : secret "insights-runtime-extractor-tls" not found Jul 14 13:28:16.389671 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:16.389651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8f731f2a-2763-4ea2-b8a3-bf991341b06c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:19.718971 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.718935 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:28:19.721833 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.721816 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:28:19.731385 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.731364 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/137eec1b-b24f-4c5d-9eb7-81601d40c532-metrics-certs\") pod \"network-metrics-daemon-sdbxt\" (UID: \"137eec1b-b24f-4c5d-9eb7-81601d40c532\") " pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:28:19.819835 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.819780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:28:19.823225 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.823207 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:28:19.832849 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.832814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d139e885-1141-43fc-a857-62e336ce02c5-original-pull-secret\") pod \"global-pull-secret-syncer-krb8t\" (UID: \"d139e885-1141-43fc-a857-62e336ce02c5\") " pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:28:19.878420 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.878391 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-krb8t" Jul 14 13:28:19.887337 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.887316 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7sfg\"" Jul 14 13:28:19.894716 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.894694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sdbxt" Jul 14 13:28:19.920936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.920898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:28:19.923277 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:19.923250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2j2c\" (UniqueName: \"kubernetes.io/projected/2ba452ed-4c7e-40ac-bbcf-da5e32db4f92-kube-api-access-b2j2c\") pod \"network-check-target-gzfsj\" (UID: \"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92\") " pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:28:20.010616 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.010591 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-krb8t"] Jul 14 13:28:20.012918 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:20.012894 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd139e885_1141_43fc_a857_62e336ce02c5.slice/crio-759a93990967d06628d91b05cc109e60baf40427df25e0d0b330d0cd78770c11 WatchSource:0}: Error finding container 759a93990967d06628d91b05cc109e60baf40427df25e0d0b330d0cd78770c11: Status 404 returned error can't find the container with id 759a93990967d06628d91b05cc109e60baf40427df25e0d0b330d0cd78770c11 Jul 14 13:28:20.027523 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.027502 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sdbxt"] Jul 14 13:28:20.029641 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:20.029623 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod137eec1b_b24f_4c5d_9eb7_81601d40c532.slice/crio-0faabce14cf8641c9564f92b2a672593a3e4d9ca3338acc4aa6c45c9293f9f0e WatchSource:0}: Error finding container 0faabce14cf8641c9564f92b2a672593a3e4d9ca3338acc4aa6c45c9293f9f0e: Status 404 returned error can't find the container with id 0faabce14cf8641c9564f92b2a672593a3e4d9ca3338acc4aa6c45c9293f9f0e Jul 14 13:28:20.132429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.132398 2568 scope.go:117] "RemoveContainer" containerID="5a661a88c8ec7ac459f00786865b51f16627f73ed5bba9dd3186087485d85df8" Jul 14 13:28:20.174707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.174686 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rgn8m\"" Jul 14 13:28:20.182249 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.182230 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:28:20.312577 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.312555 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gzfsj"] Jul 14 13:28:20.315237 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:20.315200 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba452ed_4c7e_40ac_bbcf_da5e32db4f92.slice/crio-52ba80b7b6c528f620d4446a9c742f486241c7c140662448dc728ef7d195134e WatchSource:0}: Error finding container 52ba80b7b6c528f620d4446a9c742f486241c7c140662448dc728ef7d195134e: Status 404 returned error can't find the container with id 52ba80b7b6c528f620d4446a9c742f486241c7c140662448dc728ef7d195134e Jul 14 13:28:20.436895 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.436859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdbxt" event={"ID":"137eec1b-b24f-4c5d-9eb7-81601d40c532","Type":"ContainerStarted","Data":"0faabce14cf8641c9564f92b2a672593a3e4d9ca3338acc4aa6c45c9293f9f0e"} Jul 14 13:28:20.438326 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.438302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gzfsj" event={"ID":"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92","Type":"ContainerStarted","Data":"ba18f72c55f70dabd8831d2456743f7c59690a85f2aee728c9cfe6b2b6b2ac69"} Jul 14 13:28:20.438431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.438342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gzfsj" event={"ID":"2ba452ed-4c7e-40ac-bbcf-da5e32db4f92","Type":"ContainerStarted","Data":"52ba80b7b6c528f620d4446a9c742f486241c7c140662448dc728ef7d195134e"} Jul 14 13:28:20.438431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.438379 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:28:20.439449 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.439424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krb8t" event={"ID":"d139e885-1141-43fc-a857-62e336ce02c5","Type":"ContainerStarted","Data":"759a93990967d06628d91b05cc109e60baf40427df25e0d0b330d0cd78770c11"} Jul 14 13:28:20.441204 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.441187 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:28:20.441283 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.441232 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-nrmmv" event={"ID":"94a009c8-8970-4d30-b26e-44c8f404767b","Type":"ContainerStarted","Data":"a4ff5e8aaf850056d74b12260424f40d5fa2649dd5d393bc87426913b1f1f528"} Jul 14 13:28:20.441489 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.441469 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:28:20.458253 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.458201 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gzfsj" podStartSLOduration=65.458185103 podStartE2EDuration="1m5.458185103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:20.456189844 +0000 UTC m=+66.922317398" watchObservedRunningTime="2026-07-14 13:28:20.458185103 +0000 UTC m=+66.924312638" Jul 14 13:28:20.483758 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.483645 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-nrmmv" podStartSLOduration=30.257082703000002 podStartE2EDuration="39.483627002s" totalImagesPullingTime="9.226544299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:20.482757135 +0000 UTC m=+66.948884712" watchObservedRunningTime="2026-07-14 13:28:20.483627002 +0000 UTC m=+66.949754555" Jul 14 13:28:20.543622 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.543589 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-nrmmv" Jul 14 13:28:20.829747 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.829693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:28:20.830130 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.829829 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:20.832167 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.832145 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"image-registry-6b8557cd44-8hwj2\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:20.832307 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.832287 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c31b1f5b-825c-4836-ba9f-ba304b7d4c45-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zj5sx\" (UID: \"c31b1f5b-825c-4836-ba9f-ba304b7d4c45\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:28:20.930408 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.930372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:20.930569 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.930420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:28:20.930569 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.930549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:20.930668 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.930593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:20.931251 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.931223 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-service-ca-bundle\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:20.932863 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.932839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40b314fe-62fc-4a4c-9fce-c782da4854a4-metrics-tls\") pod \"dns-default-ls9n8\" (UID: \"40b314fe-62fc-4a4c-9fce-c782da4854a4\") " pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:20.932921 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.932861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b1c003-057f-48ca-abbb-e1f6b4b3bbc9-metrics-certs\") pod \"router-default-c7b478588-9cfpm\" (UID: \"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9\") " pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:20.932960 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:20.932919 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f9e29ed3-0274-43f7-8d29-848cec6de6a9-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-swx7b\" (UID: \"f9e29ed3-0274-43f7-8d29-848cec6de6a9\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:28:21.050603 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.050566 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-szll9\"" Jul 14 13:28:21.058285 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.058254 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" Jul 14 13:28:21.063577 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.063551 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jzn49\"" Jul 14 13:28:21.071619 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.071574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:21.082866 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.082808 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hh6kq\"" Jul 14 13:28:21.090331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.090278 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:21.098759 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.098697 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dgb5n\"" Jul 14 13:28:21.106133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.105872 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" Jul 14 13:28:21.142115 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.141972 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6rczr\"" Jul 14 13:28:21.149861 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.149359 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:21.267874 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.267837 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx"] Jul 14 13:28:21.280668 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:21.279867 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc31b1f5b_825c_4836_ba9f_ba304b7d4c45.slice/crio-bed808a9e8be415d21b4c3fd2ebf51eab8de059e68fc601877a46985d4b5e789 WatchSource:0}: Error finding container bed808a9e8be415d21b4c3fd2ebf51eab8de059e68fc601877a46985d4b5e789: Status 404 returned error can't find the container with id bed808a9e8be415d21b4c3fd2ebf51eab8de059e68fc601877a46985d4b5e789 Jul 14 13:28:21.299016 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.298985 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:28:21.301515 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:21.301488 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee0bc73_1865_422b_8b11_726b3d376674.slice/crio-52c223b290db5ba1d8ae63824293e237356b661da380a0ef38df03c47d83d71a WatchSource:0}: Error finding container 52c223b290db5ba1d8ae63824293e237356b661da380a0ef38df03c47d83d71a: Status 404 returned error can't find the container with id 52c223b290db5ba1d8ae63824293e237356b661da380a0ef38df03c47d83d71a Jul 14 13:28:21.329429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.329358 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c7b478588-9cfpm"] Jul 14 13:28:21.333036 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:21.332892 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b1c003_057f_48ca_abbb_e1f6b4b3bbc9.slice/crio-eccecd6f1f0c170c832375d3946421a75c469236a0b9299427e1127e1537cd9d WatchSource:0}: Error finding container eccecd6f1f0c170c832375d3946421a75c469236a0b9299427e1127e1537cd9d: Status 404 returned error can't find the container with id eccecd6f1f0c170c832375d3946421a75c469236a0b9299427e1127e1537cd9d Jul 14 13:28:21.344783 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.344484 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b"] Jul 14 13:28:21.369824 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.369602 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ls9n8"] Jul 14 13:28:21.374184 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:21.374155 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b314fe_62fc_4a4c_9fce_c782da4854a4.slice/crio-ffcf81669f4759e5a30e03b4b03007d828f9d46d5774acb22d99592261560e9a WatchSource:0}: Error finding container ffcf81669f4759e5a30e03b4b03007d828f9d46d5774acb22d99592261560e9a: Status 404 returned error can't find the container with id ffcf81669f4759e5a30e03b4b03007d828f9d46d5774acb22d99592261560e9a Jul 14 13:28:21.448503 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.448424 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" event={"ID":"c31b1f5b-825c-4836-ba9f-ba304b7d4c45","Type":"ContainerStarted","Data":"bed808a9e8be415d21b4c3fd2ebf51eab8de059e68fc601877a46985d4b5e789"} Jul 14 13:28:21.451660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.450844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" event={"ID":"eee0bc73-1865-422b-8b11-726b3d376674","Type":"ContainerStarted","Data":"95493dfc23852991a7972d781a01fda7655e06d81bb1f1c921b7c12285f4e0b8"} Jul 14 13:28:21.451660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.450894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" event={"ID":"eee0bc73-1865-422b-8b11-726b3d376674","Type":"ContainerStarted","Data":"52c223b290db5ba1d8ae63824293e237356b661da380a0ef38df03c47d83d71a"} Jul 14 13:28:21.451660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.451602 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:21.453905 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.453845 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls9n8" event={"ID":"40b314fe-62fc-4a4c-9fce-c782da4854a4","Type":"ContainerStarted","Data":"ffcf81669f4759e5a30e03b4b03007d828f9d46d5774acb22d99592261560e9a"} Jul 14 13:28:21.455755 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.455664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c7b478588-9cfpm" event={"ID":"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9","Type":"ContainerStarted","Data":"eb80316f58f35a0fdcd6f02c75f0b3fdeb7ad720110a2584057b6a6330d62536"} Jul 14 13:28:21.455755 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.455702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c7b478588-9cfpm" event={"ID":"93b1c003-057f-48ca-abbb-e1f6b4b3bbc9","Type":"ContainerStarted","Data":"eccecd6f1f0c170c832375d3946421a75c469236a0b9299427e1127e1537cd9d"} Jul 14 13:28:21.457209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.457132 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" event={"ID":"f9e29ed3-0274-43f7-8d29-848cec6de6a9","Type":"ContainerStarted","Data":"398e715d19ed61425d821aff0d03716ce7791d3b5a650d22413fb641d456f294"} Jul 14 13:28:21.474677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.474622 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" podStartSLOduration=66.474599005 podStartE2EDuration="1m6.474599005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:21.473319291 +0000 UTC m=+67.939446864" watchObservedRunningTime="2026-07-14 13:28:21.474599005 +0000 UTC m=+67.940726564" Jul 14 13:28:21.498521 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:21.498470 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c7b478588-9cfpm" podStartSLOduration=45.49845635 podStartE2EDuration="45.49845635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:21.497273133 +0000 UTC m=+67.963400708" watchObservedRunningTime="2026-07-14 13:28:21.49845635 +0000 UTC m=+67.964583904" Jul 14 13:28:22.091303 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.091263 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:22.094098 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.094072 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:22.464861 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.464814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdbxt" event={"ID":"137eec1b-b24f-4c5d-9eb7-81601d40c532","Type":"ContainerStarted","Data":"fe1046764490b85b717fe9ae1d1b45760f269eb29c9d28efb1645821b8e4be5e"} Jul 14 13:28:22.465026 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.464891 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sdbxt" event={"ID":"137eec1b-b24f-4c5d-9eb7-81601d40c532","Type":"ContainerStarted","Data":"5965ad8a38c44e719e4ab687a8dc1c0c0b8372dd73e41a776007c7a73e9bfe52"} Jul 14 13:28:22.465691 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.465647 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:22.468119 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:22.468009 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c7b478588-9cfpm" Jul 14 13:28:23.492523 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:23.492461 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sdbxt" podStartSLOduration=67.38621113 podStartE2EDuration="1m9.492445486s" totalImagesPullingTime="2.106234356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:23.490939441 +0000 UTC m=+69.957067019" watchObservedRunningTime="2026-07-14 13:28:23.492445486 +0000 UTC m=+69.958573071" Jul 14 13:28:25.275237 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.275204 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-p6xv5"] Jul 14 13:28:25.277435 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.277397 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:25.281667 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.281404 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 13:28:25.281667 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.281529 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fzz5c\"" Jul 14 13:28:25.281994 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.281404 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 13:28:25.288810 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.288787 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-p6xv5"] Jul 14 13:28:25.373065 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.373034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxbcn\" (UniqueName: \"kubernetes.io/projected/83480c3f-ad8c-484d-9661-f75f3bd7e293-kube-api-access-sxbcn\") pod \"downloads-68c9cbb497-p6xv5\" (UID: \"83480c3f-ad8c-484d-9661-f75f3bd7e293\") " pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:25.473450 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.473404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxbcn\" (UniqueName: \"kubernetes.io/projected/83480c3f-ad8c-484d-9661-f75f3bd7e293-kube-api-access-sxbcn\") pod \"downloads-68c9cbb497-p6xv5\" (UID: \"83480c3f-ad8c-484d-9661-f75f3bd7e293\") " pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:25.487629 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.487559 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxbcn\" (UniqueName: \"kubernetes.io/projected/83480c3f-ad8c-484d-9661-f75f3bd7e293-kube-api-access-sxbcn\") pod \"downloads-68c9cbb497-p6xv5\" (UID: \"83480c3f-ad8c-484d-9661-f75f3bd7e293\") " pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:25.589397 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:25.589371 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:26.255574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.255532 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-p6xv5"] Jul 14 13:28:26.259986 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:26.259959 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83480c3f_ad8c_484d_9661_f75f3bd7e293.slice/crio-a74c55c093b1a6cb4534311bd7f81a0e93ad5f5bb1020e039583c3eb182a8ad1 WatchSource:0}: Error finding container a74c55c093b1a6cb4534311bd7f81a0e93ad5f5bb1020e039583c3eb182a8ad1: Status 404 returned error can't find the container with id a74c55c093b1a6cb4534311bd7f81a0e93ad5f5bb1020e039583c3eb182a8ad1 Jul 14 13:28:26.482270 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.482236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-krb8t" event={"ID":"d139e885-1141-43fc-a857-62e336ce02c5","Type":"ContainerStarted","Data":"f3eae2b3271e18cba7b62555f3300635f6d880e533ad9114d691a12d96ab9e8b"} Jul 14 13:28:26.484125 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.484094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls9n8" event={"ID":"40b314fe-62fc-4a4c-9fce-c782da4854a4","Type":"ContainerStarted","Data":"662a8de7e599279fd2f84ad5307fa757e57f0c4b247c7f0f8a42fba042cfe367"} Jul 14 13:28:26.484266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.484147 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ls9n8" event={"ID":"40b314fe-62fc-4a4c-9fce-c782da4854a4","Type":"ContainerStarted","Data":"de77aa36b0b98dd1ee207755a41b69e67a7edee981dce0992e4407292e900814"} Jul 14 13:28:26.484266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.484180 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:26.486006 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.485983 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" event={"ID":"f9e29ed3-0274-43f7-8d29-848cec6de6a9","Type":"ContainerStarted","Data":"bd4f4c0422aab461d25d82f029d5eb4f0ab1451b88e7a19e277dc55f64c1c937"} Jul 14 13:28:26.486116 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.486034 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" event={"ID":"f9e29ed3-0274-43f7-8d29-848cec6de6a9","Type":"ContainerStarted","Data":"b87e0d91e551d4c6433c4a417ed1a88ca09df324c23c92be47d09a6f00d7bbdf"} Jul 14 13:28:26.487116 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.487096 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-p6xv5" event={"ID":"83480c3f-ad8c-484d-9661-f75f3bd7e293","Type":"ContainerStarted","Data":"a74c55c093b1a6cb4534311bd7f81a0e93ad5f5bb1020e039583c3eb182a8ad1"} Jul 14 13:28:26.488904 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.488879 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" event={"ID":"c31b1f5b-825c-4836-ba9f-ba304b7d4c45","Type":"ContainerStarted","Data":"fc66eab5916e8d15a6a63a38ff1f848b79277765192767e89dbbf583c189c05e"} Jul 14 13:28:26.503587 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.503544 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-krb8t" podStartSLOduration=65.446704083 podStartE2EDuration="1m11.503527346s" totalImagesPullingTime="6.056823263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:26.503420342 +0000 UTC m=+72.969547887" watchObservedRunningTime="2026-07-14 13:28:26.503527346 +0000 UTC m=+72.969654901" Jul 14 13:28:26.528335 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.528289 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ls9n8" podStartSLOduration=33.839364769 podStartE2EDuration="38.528274968s" totalImagesPullingTime="4.688910199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:26.526518141 +0000 UTC m=+72.992645726" watchObservedRunningTime="2026-07-14 13:28:26.528274968 +0000 UTC m=+72.994402522" Jul 14 13:28:26.552276 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.552228 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zj5sx" podStartSLOduration=46.770697915 podStartE2EDuration="51.552213746s" totalImagesPullingTime="4.781515831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:26.550995628 +0000 UTC m=+73.017123209" watchObservedRunningTime="2026-07-14 13:28:26.552213746 +0000 UTC m=+73.018341301" Jul 14 13:28:26.570385 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.570300 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-swx7b" podStartSLOduration=41.921275531 podStartE2EDuration="46.570287028s" totalImagesPullingTime="4.649011497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:26.569622623 +0000 UTC m=+73.035750202" watchObservedRunningTime="2026-07-14 13:28:26.570287028 +0000 UTC m=+73.036414582" Jul 14 13:28:26.687154 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.687121 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m"] Jul 14 13:28:26.690295 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.690278 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:26.693221 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.693198 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 13:28:26.693352 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.693206 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tzc6c\"" Jul 14 13:28:26.698921 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.698891 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m"] Jul 14 13:28:26.785924 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.785887 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bsf6m\" (UID: \"5e7efe89-0f8c-46e9-93b7-8413a6028a5f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:26.887226 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:26.887130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bsf6m\" (UID: \"5e7efe89-0f8c-46e9-93b7-8413a6028a5f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:26.887394 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:26.887281 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 14 13:28:26.887394 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:26.887354 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates podName:5e7efe89-0f8c-46e9-93b7-8413a6028a5f nodeName:}" failed. No retries permitted until 2026-07-14 13:28:27.387336122 +0000 UTC m=+73.853463659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bsf6m" (UID: "5e7efe89-0f8c-46e9-93b7-8413a6028a5f") : secret "prometheus-operator-admission-webhook-tls" not found Jul 14 13:28:27.390947 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:27.390912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bsf6m\" (UID: \"5e7efe89-0f8c-46e9-93b7-8413a6028a5f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:27.391097 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:27.391039 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 14 13:28:27.391097 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:27.391092 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates podName:5e7efe89-0f8c-46e9-93b7-8413a6028a5f nodeName:}" failed. No retries permitted until 2026-07-14 13:28:28.391078464 +0000 UTC m=+74.857205997 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-bsf6m" (UID: "5e7efe89-0f8c-46e9-93b7-8413a6028a5f") : secret "prometheus-operator-admission-webhook-tls" not found Jul 14 13:28:28.399939 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:28.399893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bsf6m\" (UID: \"5e7efe89-0f8c-46e9-93b7-8413a6028a5f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:28.402514 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:28.402484 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e7efe89-0f8c-46e9-93b7-8413a6028a5f-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-bsf6m\" (UID: \"5e7efe89-0f8c-46e9-93b7-8413a6028a5f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:28.500715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:28.500684 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:28.643583 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:28.643549 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m"] Jul 14 13:28:28.646452 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:28.646411 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7efe89_0f8c_46e9_93b7_8413a6028a5f.slice/crio-2631e20c67477fd0e9f9d1567dc0783368bf16b0a76f066e7fcd441b2d1ec4a5 WatchSource:0}: Error finding container 2631e20c67477fd0e9f9d1567dc0783368bf16b0a76f066e7fcd441b2d1ec4a5: Status 404 returned error can't find the container with id 2631e20c67477fd0e9f9d1567dc0783368bf16b0a76f066e7fcd441b2d1ec4a5 Jul 14 13:28:29.503988 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:29.503940 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" event={"ID":"5e7efe89-0f8c-46e9-93b7-8413a6028a5f","Type":"ContainerStarted","Data":"2631e20c67477fd0e9f9d1567dc0783368bf16b0a76f066e7fcd441b2d1ec4a5"} Jul 14 13:28:31.512808 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:31.512778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" event={"ID":"5e7efe89-0f8c-46e9-93b7-8413a6028a5f","Type":"ContainerStarted","Data":"ff3c883bf9487bfdbabac13b671c185cc85f4c76941fffa4986a5b3b3cf9b5b0"} Jul 14 13:28:31.513191 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:31.512953 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:31.517821 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:31.517797 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" Jul 14 13:28:31.532681 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:31.532645 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-bsf6m" podStartSLOduration=3.6234352359999997 podStartE2EDuration="5.532630967s" totalImagesPullingTime="1.909195731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:31.531796617 +0000 UTC m=+77.997924192" watchObservedRunningTime="2026-07-14 13:28:31.532630967 +0000 UTC m=+77.998758521" Jul 14 13:28:32.433715 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:32.433680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:32.436209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:32.436181 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8f731f2a-2763-4ea2-b8a3-bf991341b06c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lwqjt\" (UID: \"8f731f2a-2763-4ea2-b8a3-bf991341b06c\") " pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:32.679843 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:32.679814 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tbwpd\"" Jul 14 13:28:32.687296 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:32.687205 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lwqjt" Jul 14 13:28:32.842506 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:32.842478 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lwqjt"] Jul 14 13:28:32.845747 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:32.845706 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f731f2a_2763_4ea2_b8a3_bf991341b06c.slice/crio-710e05c490f20bd245673e661559a3046a1748be7c3283bbfa0919456d13035c WatchSource:0}: Error finding container 710e05c490f20bd245673e661559a3046a1748be7c3283bbfa0919456d13035c: Status 404 returned error can't find the container with id 710e05c490f20bd245673e661559a3046a1748be7c3283bbfa0919456d13035c Jul 14 13:28:33.524084 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:33.524010 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lwqjt" event={"ID":"8f731f2a-2763-4ea2-b8a3-bf991341b06c","Type":"ContainerStarted","Data":"710e05c490f20bd245673e661559a3046a1748be7c3283bbfa0919456d13035c"} Jul 14 13:28:36.497046 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:36.496909 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ls9n8" Jul 14 13:28:37.543798 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:37.543759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lwqjt" event={"ID":"8f731f2a-2763-4ea2-b8a3-bf991341b06c","Type":"ContainerStarted","Data":"405395a025793c2e72869419eafecfd91f520ca22207ee9d5a2e8ddbd0dcd970"} Jul 14 13:28:38.550028 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:38.549999 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lwqjt" event={"ID":"8f731f2a-2763-4ea2-b8a3-bf991341b06c","Type":"ContainerStarted","Data":"d3dea48d80f02bdb888de13a47cf18183245ab47eb10147da711a4a471fe855b"} Jul 14 13:28:39.122270 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.121865 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6"] Jul 14 13:28:39.125059 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.125034 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.129504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.129275 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 13:28:39.129504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.129298 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 13:28:39.129504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.129359 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-z6q2t\"" Jul 14 13:28:39.129504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.129300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 13:28:39.137482 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.137457 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6"] Jul 14 13:28:39.142774 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.142722 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nwc8k"] Jul 14 13:28:39.145545 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.145425 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.148443 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.148425 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 13:28:39.148541 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.148463 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 13:28:39.148710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.148691 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 13:28:39.149358 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.149193 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zg27v\"" Jul 14 13:28:39.199084 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199049 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-root\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-wtmp\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4mlf\" (UniqueName: \"kubernetes.io/projected/c3613939-6924-408d-af40-84bc02cd7c43-kube-api-access-h4mlf\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199145 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-sys\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199173 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx26z\" (UniqueName: \"kubernetes.io/projected/97294527-1356-4ac2-89ce-f05db1255f12-kube-api-access-nx26z\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199201 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-tls\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199589 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.199589 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199383 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.199589 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199589 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3613939-6924-408d-af40-84bc02cd7c43-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.199801 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-textfile\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.199801 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.199677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-metrics-client-ca\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.300452 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx26z\" (UniqueName: \"kubernetes.io/projected/97294527-1356-4ac2-89ce-f05db1255f12-kube-api-access-nx26z\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.300644 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300468 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-tls\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.300644 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.300884 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.300978 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.300978 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.301081 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.300995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3613939-6924-408d-af40-84bc02cd7c43-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.301137 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:39.301086 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 14 13:28:39.301236 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:39.301181 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls podName:c3613939-6924-408d-af40-84bc02cd7c43 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:39.801158774 +0000 UTC m=+86.267286327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-czcl6" (UID: "c3613939-6924-408d-af40-84bc02cd7c43") : secret "openshift-state-metrics-tls" not found Jul 14 13:28:39.301657 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.301622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-textfile\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.301773 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.301664 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-accelerators-collector-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.302763 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.301767 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3613939-6924-408d-af40-84bc02cd7c43-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.303045 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.302956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-metrics-client-ca\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.303415 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-textfile\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.303444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-root\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.303514 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-root\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.304195 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.304262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97294527-1356-4ac2-89ce-f05db1255f12-metrics-client-ca\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.304497 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-wtmp\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.306032 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-tls\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.311079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx26z\" (UniqueName: \"kubernetes.io/projected/97294527-1356-4ac2-89ce-f05db1255f12-kube-api-access-nx26z\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.303558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-node-exporter-wtmp\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.311152 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4mlf\" (UniqueName: \"kubernetes.io/projected/c3613939-6924-408d-af40-84bc02cd7c43-kube-api-access-h4mlf\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.311184 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-sys\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.311500 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.311272 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97294527-1356-4ac2-89ce-f05db1255f12-sys\") pod \"node-exporter-nwc8k\" (UID: \"97294527-1356-4ac2-89ce-f05db1255f12\") " pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.313749 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.313041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.325320 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.325295 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4mlf\" (UniqueName: \"kubernetes.io/projected/c3613939-6924-408d-af40-84bc02cd7c43-kube-api-access-h4mlf\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.457082 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.456999 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nwc8k" Jul 14 13:28:39.468662 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:39.468617 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97294527_1356_4ac2_89ce_f05db1255f12.slice/crio-6467935d218937bd453af3ce36367f1b1e91ec77f34e9fe54ccec60a1f39ff28 WatchSource:0}: Error finding container 6467935d218937bd453af3ce36367f1b1e91ec77f34e9fe54ccec60a1f39ff28: Status 404 returned error can't find the container with id 6467935d218937bd453af3ce36367f1b1e91ec77f34e9fe54ccec60a1f39ff28 Jul 14 13:28:39.555559 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.555504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwc8k" event={"ID":"97294527-1356-4ac2-89ce-f05db1255f12","Type":"ContainerStarted","Data":"6467935d218937bd453af3ce36367f1b1e91ec77f34e9fe54ccec60a1f39ff28"} Jul 14 13:28:39.817183 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:39.817015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:39.817183 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:39.817170 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 14 13:28:39.817426 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:28:39.817263 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls podName:c3613939-6924-408d-af40-84bc02cd7c43 nodeName:}" failed. No retries permitted until 2026-07-14 13:28:40.817240896 +0000 UTC m=+87.283368447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-czcl6" (UID: "c3613939-6924-408d-af40-84bc02cd7c43") : secret "openshift-state-metrics-tls" not found Jul 14 13:28:40.223587 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.223503 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:28:40.230583 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.230558 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.233855 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.233829 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 13:28:40.233855 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.233838 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 13:28:40.234174 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234098 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 13:28:40.234285 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234178 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 13:28:40.234285 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234099 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 13:28:40.234444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234430 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 13:28:40.234510 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234469 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5ssn4\"" Jul 14 13:28:40.234567 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234555 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 13:28:40.234658 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.234521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 13:28:40.235395 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.235376 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 13:28:40.243536 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.243451 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:28:40.321661 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.321856 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.321856 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.321856 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.321856 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321950 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.321988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8blsr\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.322023 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322321 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.322107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322321 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.322204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.322321 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.322235 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423150 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423324 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423324 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423231 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423324 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423274 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423324 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423303 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blsr\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423455 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423563 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.423829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.423765 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.425325 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.425294 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.426196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.426155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.428622 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.428435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.428622 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.428593 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.429056 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.429008 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.429178 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.429058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.429178 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.428792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.429479 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.429437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.430548 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.430495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.430548 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.430502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.430992 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.430966 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.434032 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.434008 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blsr\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr\") pod \"alertmanager-main-0\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.544026 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.543989 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:28:40.828185 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.828109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:40.830710 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.830682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3613939-6924-408d-af40-84bc02cd7c43-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-czcl6\" (UID: \"c3613939-6924-408d-af40-84bc02cd7c43\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:40.937518 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:40.937495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" Jul 14 13:28:41.074917 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.074822 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:28:41.075146 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:41.075114 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff6b544_ae62_49e7_bdae_6085428d8d33.slice/crio-2712ed11531b43ee66e9e310374c2721598000bc53c41d0071af10591346325a WatchSource:0}: Error finding container 2712ed11531b43ee66e9e310374c2721598000bc53c41d0071af10591346325a: Status 404 returned error can't find the container with id 2712ed11531b43ee66e9e310374c2721598000bc53c41d0071af10591346325a Jul 14 13:28:41.079558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.079439 2568 patch_prober.go:28] interesting pod/image-registry-6b8557cd44-8hwj2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 13:28:41.079558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.079491 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" podUID="eee0bc73-1865-422b-8b11-726b3d376674" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 13:28:41.115514 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.115419 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6"] Jul 14 13:28:41.118757 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:41.118714 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3613939_6924_408d_af40_84bc02cd7c43.slice/crio-1f5f4830a72c442c45b8116bbccb1a303b0ff714ae8dafc9b65a4e7ad0c2e41a WatchSource:0}: Error finding container 1f5f4830a72c442c45b8116bbccb1a303b0ff714ae8dafc9b65a4e7ad0c2e41a: Status 404 returned error can't find the container with id 1f5f4830a72c442c45b8116bbccb1a303b0ff714ae8dafc9b65a4e7ad0c2e41a Jul 14 13:28:41.242075 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.241986 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c654f798b-8fmtg"] Jul 14 13:28:41.246082 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.246058 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.249157 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6rhooc7pt4oi5\"" Jul 14 13:28:41.249157 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249152 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 13:28:41.249378 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 13:28:41.249378 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249189 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 13:28:41.249378 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249262 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 13:28:41.249378 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-cdtbz\"" Jul 14 13:28:41.249602 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.249585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 13:28:41.263266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.263222 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c654f798b-8fmtg"] Jul 14 13:28:41.332185 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332087 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332185 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332159 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332406 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332202 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332406 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332406 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a185f3ef-c1a2-40c3-b0cc-760931805d06-metrics-client-ca\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332559 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332424 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332559 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncbgg\" (UniqueName: \"kubernetes.io/projected/a185f3ef-c1a2-40c3-b0cc-760931805d06-kube-api-access-ncbgg\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.332559 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.332511 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-grpc-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433612 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433813 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncbgg\" (UniqueName: \"kubernetes.io/projected/a185f3ef-c1a2-40c3-b0cc-760931805d06-kube-api-access-ncbgg\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433813 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-grpc-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433813 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433813 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433767 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.433813 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433799 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.434058 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433838 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.434058 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.433882 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a185f3ef-c1a2-40c3-b0cc-760931805d06-metrics-client-ca\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.434815 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.434757 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a185f3ef-c1a2-40c3-b0cc-760931805d06-metrics-client-ca\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.437216 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.437141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.438201 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.437829 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-grpc-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.438201 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.438053 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-tls\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.438385 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.438230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.438385 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.438359 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.439464 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.439437 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a185f3ef-c1a2-40c3-b0cc-760931805d06-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.444693 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.444673 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncbgg\" (UniqueName: \"kubernetes.io/projected/a185f3ef-c1a2-40c3-b0cc-760931805d06-kube-api-access-ncbgg\") pod \"thanos-querier-7c654f798b-8fmtg\" (UID: \"a185f3ef-c1a2-40c3-b0cc-760931805d06\") " pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.560219 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.560169 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:41.565807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.565775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"2712ed11531b43ee66e9e310374c2721598000bc53c41d0071af10591346325a"} Jul 14 13:28:41.568331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.568308 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lwqjt" event={"ID":"8f731f2a-2763-4ea2-b8a3-bf991341b06c","Type":"ContainerStarted","Data":"c2667acce3e1207b38b13e076a2f8f21a2f8c07bbfd3bf63717f63a6721b3040"} Jul 14 13:28:41.570496 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.570474 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" event={"ID":"c3613939-6924-408d-af40-84bc02cd7c43","Type":"ContainerStarted","Data":"e5a9c5c0bfd1b8874295841f61d4b896b72e66ec388d59f00f3f50cbf220b385"} Jul 14 13:28:41.570580 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.570528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" event={"ID":"c3613939-6924-408d-af40-84bc02cd7c43","Type":"ContainerStarted","Data":"081c8f933abb6c3cc3cb0155abe3c16bb9a6b9b16bf16719c4d3dd7bcd575aa8"} Jul 14 13:28:41.570580 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.570559 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" event={"ID":"c3613939-6924-408d-af40-84bc02cd7c43","Type":"ContainerStarted","Data":"1f5f4830a72c442c45b8116bbccb1a303b0ff714ae8dafc9b65a4e7ad0c2e41a"} Jul 14 13:28:41.592822 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:41.592710 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lwqjt" podStartSLOduration=33.652535914 podStartE2EDuration="41.592694682s" totalImagesPullingTime="7.940158768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:28:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:41.591854997 +0000 UTC m=+88.057982572" watchObservedRunningTime="2026-07-14 13:28:41.592694682 +0000 UTC m=+88.058822236" Jul 14 13:28:43.475694 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.475665 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:28:43.657173 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.657140 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85f8b5cb44-lhjkl"] Jul 14 13:28:43.659740 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.659703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.663164 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663113 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 13:28:43.663164 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663152 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 13:28:43.663329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663178 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5zsgh\"" Jul 14 13:28:43.663329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663116 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 13:28:43.663509 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663492 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2fbh4t5clnsv0\"" Jul 14 13:28:43.663854 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.663833 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 13:28:43.673492 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.673469 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85f8b5cb44-lhjkl"] Jul 14 13:28:43.756359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756527 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfwtj\" (UniqueName: \"kubernetes.io/projected/5569d118-8f6d-40c5-8fc9-7a5e563987fd-kube-api-access-bfwtj\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756527 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-client-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756645 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756533 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-tls\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756645 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5569d118-8f6d-40c5-8fc9-7a5e563987fd-audit-log\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756645 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756629 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-client-certs\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.756816 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.756663 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-metrics-server-audit-profiles\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-tls\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5569d118-8f6d-40c5-8fc9-7a5e563987fd-audit-log\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857420 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-client-certs\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-metrics-server-audit-profiles\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857530 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfwtj\" (UniqueName: \"kubernetes.io/projected/5569d118-8f6d-40c5-8fc9-7a5e563987fd-kube-api-access-bfwtj\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.858147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.857646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-client-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.862807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.859160 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5569d118-8f6d-40c5-8fc9-7a5e563987fd-audit-log\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.862807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.859510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.862807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.860693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-client-ca-bundle\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.862807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.862704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5569d118-8f6d-40c5-8fc9-7a5e563987fd-metrics-server-audit-profiles\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.862807 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.862782 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-tls\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.863976 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.863926 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5569d118-8f6d-40c5-8fc9-7a5e563987fd-secret-metrics-server-client-certs\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.871937 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.871890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfwtj\" (UniqueName: \"kubernetes.io/projected/5569d118-8f6d-40c5-8fc9-7a5e563987fd-kube-api-access-bfwtj\") pod \"metrics-server-85f8b5cb44-lhjkl\" (UID: \"5569d118-8f6d-40c5-8fc9-7a5e563987fd\") " pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:43.918695 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.918663 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g"] Jul 14 13:28:43.921542 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.921522 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:43.924608 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.924585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 14 13:28:43.924740 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.924636 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6ndvr\"" Jul 14 13:28:43.933763 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.933743 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g"] Jul 14 13:28:43.946287 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.946262 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:28:43.948891 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.948868 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:43.951960 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.951929 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 13:28:43.952088 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.952021 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 13:28:43.952222 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.952202 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-xz9wp\"" Jul 14 13:28:43.952651 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.952474 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 13:28:43.952651 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.952486 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 13:28:43.952651 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.952480 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 13:28:43.961925 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.961873 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 13:28:43.962929 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.962904 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:28:43.971432 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:43.971409 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:28:44.060571 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060791 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060585 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjzc8\" (UniqueName: \"kubernetes.io/projected/1ce5a003-11b9-4045-91a2-6991a504d8aa-kube-api-access-pjzc8\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.060791 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060616 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ce5a003-11b9-4045-91a2-6991a504d8aa-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.060791 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060898 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.060990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.060963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw2hc\" (UniqueName: \"kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.161859 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.161807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.161874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.161910 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.161945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.161974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162036 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162001 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw2hc\" (UniqueName: \"kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjzc8\" (UniqueName: \"kubernetes.io/projected/1ce5a003-11b9-4045-91a2-6991a504d8aa-kube-api-access-pjzc8\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.162289 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ce5a003-11b9-4045-91a2-6991a504d8aa-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.162798 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162744 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162928 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162840 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162928 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.162928 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.162869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.164943 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.164918 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1ce5a003-11b9-4045-91a2-6991a504d8aa-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.165262 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.165243 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.165359 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.165343 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.171855 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.171833 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjzc8\" (UniqueName: \"kubernetes.io/projected/1ce5a003-11b9-4045-91a2-6991a504d8aa-kube-api-access-pjzc8\") pod \"monitoring-plugin-5f68df7f58-wtd5g\" (UID: \"1ce5a003-11b9-4045-91a2-6991a504d8aa\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.172032 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.172003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw2hc\" (UniqueName: \"kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc\") pod \"console-8655ff76dd-xfbcd\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:44.233771 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.233738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:44.265681 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:44.265637 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:28:47.545365 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:47.545328 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:28:48.606999 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:48.606971 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:28:49.093576 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.093415 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g"] Jul 14 13:28:49.098271 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:49.098237 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce5a003_11b9_4045_91a2_6991a504d8aa.slice/crio-a2d5fbe1e0d50b19bbb61d0f6a249d1e87a78516abcf9719204e82b8712e77b8 WatchSource:0}: Error finding container a2d5fbe1e0d50b19bbb61d0f6a249d1e87a78516abcf9719204e82b8712e77b8: Status 404 returned error can't find the container with id a2d5fbe1e0d50b19bbb61d0f6a249d1e87a78516abcf9719204e82b8712e77b8 Jul 14 13:28:49.119922 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.119434 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c654f798b-8fmtg"] Jul 14 13:28:49.121598 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:49.121567 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda185f3ef_c1a2_40c3_b0cc_760931805d06.slice/crio-c9b00625e8780f686ec5a553710cdc15c2533234ff2d6a127fda052ed8024365 WatchSource:0}: Error finding container c9b00625e8780f686ec5a553710cdc15c2533234ff2d6a127fda052ed8024365: Status 404 returned error can't find the container with id c9b00625e8780f686ec5a553710cdc15c2533234ff2d6a127fda052ed8024365 Jul 14 13:28:49.142966 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.141962 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85f8b5cb44-lhjkl"] Jul 14 13:28:49.147247 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:49.147213 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5569d118_8f6d_40c5_8fc9_7a5e563987fd.slice/crio-6a9c0d85bcb540c348fb33b62875b1f14fb354ab87e60eeb08bd74cd4ddfe270 WatchSource:0}: Error finding container 6a9c0d85bcb540c348fb33b62875b1f14fb354ab87e60eeb08bd74cd4ddfe270: Status 404 returned error can't find the container with id 6a9c0d85bcb540c348fb33b62875b1f14fb354ab87e60eeb08bd74cd4ddfe270 Jul 14 13:28:49.344607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.344572 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:28:49.613358 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.613317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" event={"ID":"1ce5a003-11b9-4045-91a2-6991a504d8aa","Type":"ContainerStarted","Data":"a2d5fbe1e0d50b19bbb61d0f6a249d1e87a78516abcf9719204e82b8712e77b8"} Jul 14 13:28:49.614808 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.614773 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" event={"ID":"5569d118-8f6d-40c5-8fc9-7a5e563987fd","Type":"ContainerStarted","Data":"6a9c0d85bcb540c348fb33b62875b1f14fb354ab87e60eeb08bd74cd4ddfe270"} Jul 14 13:28:49.616810 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.616783 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-p6xv5" event={"ID":"83480c3f-ad8c-484d-9661-f75f3bd7e293","Type":"ContainerStarted","Data":"659923cea63d594b95b265c97af6b3c4c1c1b38227d496e22068cf268055e21f"} Jul 14 13:28:49.617241 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.617220 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:49.621968 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.621550 2568 generic.go:356] "Generic (PLEG): container finished" podID="97294527-1356-4ac2-89ce-f05db1255f12" containerID="81138ffad897e15fe7900eb551ec37ab85feadaf4d91066ba69c20f7d23899b7" exitCode=0 Jul 14 13:28:49.621968 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.621626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwc8k" event={"ID":"97294527-1356-4ac2-89ce-f05db1255f12","Type":"ContainerDied","Data":"81138ffad897e15fe7900eb551ec37ab85feadaf4d91066ba69c20f7d23899b7"} Jul 14 13:28:49.622332 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.622020 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-p6xv5" Jul 14 13:28:49.624457 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.624428 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"c9b00625e8780f686ec5a553710cdc15c2533234ff2d6a127fda052ed8024365"} Jul 14 13:28:49.636714 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:49.636621 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-p6xv5" podStartSLOduration=1.9520354850000001 podStartE2EDuration="24.63660045s" totalImagesPullingTime="22.684564965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:49.636139643 +0000 UTC m=+96.102267226" watchObservedRunningTime="2026-07-14 13:28:49.63660045 +0000 UTC m=+96.102728015" Jul 14 13:28:49.918421 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:28:49.918133 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2f2d25_0ff3_4773_a9d7_8dcb495bc7da.slice/crio-fb81c2886430d337840120ec243f4af215253f00fa4d6a2ebb7afa7d6a301fcc WatchSource:0}: Error finding container fb81c2886430d337840120ec243f4af215253f00fa4d6a2ebb7afa7d6a301fcc: Status 404 returned error can't find the container with id fb81c2886430d337840120ec243f4af215253f00fa4d6a2ebb7afa7d6a301fcc Jul 14 13:28:50.638942 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.638851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" event={"ID":"c3613939-6924-408d-af40-84bc02cd7c43","Type":"ContainerStarted","Data":"f4e478ead20585828892c0b1ded9500f6341155d27907af26082e93c052f0746"} Jul 14 13:28:50.646483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.645667 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwc8k" event={"ID":"97294527-1356-4ac2-89ce-f05db1255f12","Type":"ContainerStarted","Data":"341382971ded0a5b139cf989521dc5056965bdf393f5c3b8fba7c1ff70a6f1f8"} Jul 14 13:28:50.646483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.645763 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nwc8k" event={"ID":"97294527-1356-4ac2-89ce-f05db1255f12","Type":"ContainerStarted","Data":"c74e6b3106975a59ee7f170f748fb39117e21350b845debb93d4a901e69af8bb"} Jul 14 13:28:50.649189 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.649113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8655ff76dd-xfbcd" event={"ID":"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da","Type":"ContainerStarted","Data":"fb81c2886430d337840120ec243f4af215253f00fa4d6a2ebb7afa7d6a301fcc"} Jul 14 13:28:50.654829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.653541 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09" exitCode=0 Jul 14 13:28:50.654829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.654751 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09"} Jul 14 13:28:50.664753 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.663325 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-czcl6" podStartSLOduration=2.818651074 podStartE2EDuration="11.663304061s" totalImagesPullingTime="8.844652987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:50.662256566 +0000 UTC m=+97.128384148" watchObservedRunningTime="2026-07-14 13:28:50.663304061 +0000 UTC m=+97.129431617" Jul 14 13:28:50.694383 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:50.692336 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nwc8k" podStartSLOduration=2.174156226 podStartE2EDuration="11.692312268s" totalImagesPullingTime="9.39996286s" totalInitContainerRuntime="118.193182ms" isStatefulPod=true podCreationTimestamp="2026-07-14 13:28:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:50.691186659 +0000 UTC m=+97.157314215" watchObservedRunningTime="2026-07-14 13:28:50.692312268 +0000 UTC m=+97.158439839" Jul 14 13:28:51.461543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:51.460688 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gzfsj" Jul 14 13:28:56.700084 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.700047 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f"} Jul 14 13:28:56.701812 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.701777 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" event={"ID":"1ce5a003-11b9-4045-91a2-6991a504d8aa","Type":"ContainerStarted","Data":"c8af62541931fe726d67b1c8f0badac5a7619b63dd6b67b690ee34a390394e5f"} Jul 14 13:28:56.703644 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.703605 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" event={"ID":"5569d118-8f6d-40c5-8fc9-7a5e563987fd","Type":"ContainerStarted","Data":"fabab343d390a0f0965acb637b25da6c464a31c24960bebad46945eeb81eb1ae"} Jul 14 13:28:56.706113 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.705843 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:56.707242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.707094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"ed2d9aeab81b4acb88b55cfb7e509300f820f11ce55ff845d3407dd07f979862"} Jul 14 13:28:56.707242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.707162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"d47e0833966e37cd670f6fc00659b332a7034798655d3c051fc14145d59d401d"} Jul 14 13:28:56.710505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.710026 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8655ff76dd-xfbcd" event={"ID":"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da","Type":"ContainerStarted","Data":"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2"} Jul 14 13:28:56.717207 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.717103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" Jul 14 13:28:56.733923 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.731852 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" podStartSLOduration=6.736069212 podStartE2EDuration="13.731814499s" totalImagesPullingTime="6.995745287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:56.731488678 +0000 UTC m=+103.197616257" watchObservedRunningTime="2026-07-14 13:28:56.731814499 +0000 UTC m=+103.197942055" Jul 14 13:28:56.750782 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.750676 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-wtd5g" podStartSLOduration=6.706321492 podStartE2EDuration="13.750656058s" totalImagesPullingTime="7.044334566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:56.749649801 +0000 UTC m=+103.215777355" watchObservedRunningTime="2026-07-14 13:28:56.750656058 +0000 UTC m=+103.216783613" Jul 14 13:28:56.771405 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:56.771349 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8655ff76dd-xfbcd" podStartSLOduration=7.717765116 podStartE2EDuration="13.771325574s" totalImagesPullingTime="6.053560458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:56.769986338 +0000 UTC m=+103.236113918" watchObservedRunningTime="2026-07-14 13:28:56.771325574 +0000 UTC m=+103.237453129" Jul 14 13:28:57.719463 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:57.719426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398"} Jul 14 13:28:57.719967 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:57.719507 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa"} Jul 14 13:28:57.719967 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:57.719550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142"} Jul 14 13:28:57.719967 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:57.719590 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf"} Jul 14 13:28:57.721873 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:57.721792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"a34d9ddd7b539f6c51b0780089d4f63937289e6d85babeb4ca07ac96fd949ef8"} Jul 14 13:28:58.730322 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:58.730286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerStarted","Data":"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76"} Jul 14 13:28:58.733390 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:58.733368 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"8d08d17081f4ff49916789a8ac3bb8430c10361c7967912935375a4ae0014804"} Jul 14 13:28:58.763590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:58.763532 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.601556931 podStartE2EDuration="18.763512204s" totalImagesPullingTime="16.131201007s" totalInitContainerRuntime="30.754266ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:58.761390759 +0000 UTC m=+105.227518357" watchObservedRunningTime="2026-07-14 13:28:58.763512204 +0000 UTC m=+105.229639758" Jul 14 13:28:59.740721 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:59.740685 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"27d5e2b996fcf4689505544dd12cc686b2a40a0e8b0469fc013f2c87f752717b"} Jul 14 13:28:59.741267 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:59.740765 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" event={"ID":"a185f3ef-c1a2-40c3-b0cc-760931805d06","Type":"ContainerStarted","Data":"b60ca91845412653789f8e7c3df3f819bebf272ff2bccf6458276c2da621af94"} Jul 14 13:28:59.741348 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:59.741278 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:28:59.771596 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:28:59.771543 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" podStartSLOduration=10.066672927 podStartE2EDuration="18.77152677s" totalImagesPullingTime="8.704853843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:28:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:28:59.770101418 +0000 UTC m=+106.236229021" watchObservedRunningTime="2026-07-14 13:28:59.77152677 +0000 UTC m=+106.237654323" Jul 14 13:29:01.756199 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:01.756174 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c654f798b-8fmtg" Jul 14 13:29:03.971698 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:03.971661 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:29:03.972122 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:03.971724 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:29:04.266925 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:04.266839 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:29:05.768571 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:05.768544 2568 generic.go:356] "Generic (PLEG): container finished" podID="7a7b5ed4-167d-4a0c-945b-b70e878a8026" containerID="728a096552c09ff7872b4f4bb24636271201381492fa5e141502315af85b8f9e" exitCode=0 Jul 14 13:29:05.769039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:05.768579 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" event={"ID":"7a7b5ed4-167d-4a0c-945b-b70e878a8026","Type":"ContainerDied","Data":"728a096552c09ff7872b4f4bb24636271201381492fa5e141502315af85b8f9e"} Jul 14 13:29:05.769039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:05.768875 2568 scope.go:117] "RemoveContainer" containerID="728a096552c09ff7872b4f4bb24636271201381492fa5e141502315af85b8f9e" Jul 14 13:29:06.774611 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:06.774570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-wdvzj" event={"ID":"7a7b5ed4-167d-4a0c-945b-b70e878a8026","Type":"ContainerStarted","Data":"1663eb1ff45a02e138489f2d7ebd518fcc3d469233495e53d5b07adfa4012d84"} Jul 14 13:29:09.787701 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:09.787666 2568 generic.go:356] "Generic (PLEG): container finished" podID="38ec00a8-80ff-4fde-a998-e065e0169859" containerID="eb3edd47381e810ad719651a9ea3ca1428ee1e88e486c12734fd71bd12650caa" exitCode=0 Jul 14 13:29:09.788103 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:09.787757 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" event={"ID":"38ec00a8-80ff-4fde-a998-e065e0169859","Type":"ContainerDied","Data":"eb3edd47381e810ad719651a9ea3ca1428ee1e88e486c12734fd71bd12650caa"} Jul 14 13:29:09.788146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:09.788109 2568 scope.go:117] "RemoveContainer" containerID="eb3edd47381e810ad719651a9ea3ca1428ee1e88e486c12734fd71bd12650caa" Jul 14 13:29:10.794084 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:10.794051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-ml48v" event={"ID":"38ec00a8-80ff-4fde-a998-e065e0169859","Type":"ContainerStarted","Data":"02e22412dc10386d9a175af9e98dc757dfe8ff1f0d06c5054ae06872a36e838b"} Jul 14 13:29:12.572463 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.572389 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" podUID="eee0bc73-1865-422b-8b11-726b3d376674" containerName="registry" containerID="cri-o://95493dfc23852991a7972d781a01fda7655e06d81bb1f1c921b7c12285f4e0b8" gracePeriod=30 Jul 14 13:29:12.809385 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.809353 2568 generic.go:356] "Generic (PLEG): container finished" podID="eee0bc73-1865-422b-8b11-726b3d376674" containerID="95493dfc23852991a7972d781a01fda7655e06d81bb1f1c921b7c12285f4e0b8" exitCode=0 Jul 14 13:29:12.809543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.809392 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" event={"ID":"eee0bc73-1865-422b-8b11-726b3d376674","Type":"ContainerDied","Data":"95493dfc23852991a7972d781a01fda7655e06d81bb1f1c921b7c12285f4e0b8"} Jul 14 13:29:12.861666 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.861628 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:29:12.956750 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956689 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956765 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq977\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956797 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956814 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956832 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956884 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.956940 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956935 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.957253 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.956981 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") pod \"eee0bc73-1865-422b-8b11-726b3d376674\" (UID: \"eee0bc73-1865-422b-8b11-726b3d376674\") " Jul 14 13:29:12.957253 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.957129 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:12.957466 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.957432 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-registry-certificates\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:12.957787 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.957759 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:12.959528 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.959482 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:29:12.959709 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.959678 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:29:12.959804 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.959706 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:29:12.959804 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.959769 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977" (OuterVolumeSpecName: "kube-api-access-fq977") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "kube-api-access-fq977". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:29:12.960047 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.960030 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:29:12.965087 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:12.965061 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "eee0bc73-1865-422b-8b11-726b3d376674" (UID: "eee0bc73-1865-422b-8b11-726b3d376674"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:29:13.058431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058391 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq977\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-kube-api-access-fq977\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058426 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-image-registry-private-configuration\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058441 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eee0bc73-1865-422b-8b11-726b3d376674-trusted-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058453 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eee0bc73-1865-422b-8b11-726b3d376674-installation-pull-secrets\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058465 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eee0bc73-1865-422b-8b11-726b3d376674-ca-trust-extracted\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058477 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-bound-sa-token\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.058674 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.058487 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eee0bc73-1865-422b-8b11-726b3d376674-registry-tls\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:13.814703 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.814673 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" Jul 14 13:29:13.814703 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.814679 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b8557cd44-8hwj2" event={"ID":"eee0bc73-1865-422b-8b11-726b3d376674","Type":"ContainerDied","Data":"52c223b290db5ba1d8ae63824293e237356b661da380a0ef38df03c47d83d71a"} Jul 14 13:29:13.815242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.814757 2568 scope.go:117] "RemoveContainer" containerID="95493dfc23852991a7972d781a01fda7655e06d81bb1f1c921b7c12285f4e0b8" Jul 14 13:29:13.843696 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.843665 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:29:13.851692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:13.851668 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b8557cd44-8hwj2"] Jul 14 13:29:14.141276 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:14.141203 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee0bc73-1865-422b-8b11-726b3d376674" path="/var/lib/kubelet/pods/eee0bc73-1865-422b-8b11-726b3d376674/volumes" Jul 14 13:29:21.740319 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:21.740254 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8655ff76dd-xfbcd" podUID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" containerName="console" containerID="cri-o://92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2" gracePeriod=15 Jul 14 13:29:22.004370 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.004307 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143220 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143281 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143378 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143404 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143490 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw2hc\" (UniqueName: \"kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.143518 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert\") pod \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\" (UID: \"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da\") " Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.144054 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.144342 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config" (OuterVolumeSpecName: "console-config") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:22.144868 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.144572 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:22.146361 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.146206 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:29:22.148231 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.148197 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:29:22.148328 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.148293 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc" (OuterVolumeSpecName: "kube-api-access-rw2hc") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "kube-api-access-rw2hc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:29:22.148561 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.148536 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" (UID: "7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244755 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw2hc\" (UniqueName: \"kubernetes.io/projected/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-kube-api-access-rw2hc\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244798 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-oauth-serving-cert\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244814 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-config\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244828 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244842 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-oauth-config\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244856 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-console-serving-cert\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.247566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.244869 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da-trusted-ca-bundle\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:29:22.853052 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.853020 2568 generic.go:356] "Generic (PLEG): container finished" podID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" containerID="92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2" exitCode=0 Jul 14 13:29:22.853487 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.853065 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8655ff76dd-xfbcd" Jul 14 13:29:22.853487 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.853106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8655ff76dd-xfbcd" event={"ID":"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da","Type":"ContainerDied","Data":"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2"} Jul 14 13:29:22.853487 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.853178 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8655ff76dd-xfbcd" event={"ID":"7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da","Type":"ContainerDied","Data":"fb81c2886430d337840120ec243f4af215253f00fa4d6a2ebb7afa7d6a301fcc"} Jul 14 13:29:22.853487 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.853209 2568 scope.go:117] "RemoveContainer" containerID="92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2" Jul 14 13:29:22.863312 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.863292 2568 scope.go:117] "RemoveContainer" containerID="92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2" Jul 14 13:29:22.863593 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:29:22.863572 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2\": container with ID starting with 92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2 not found: ID does not exist" containerID="92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2" Jul 14 13:29:22.863649 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.863602 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2"} err="failed to get container status \"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2\": rpc error: code = NotFound desc = could not find container \"92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2\": container with ID starting with 92265add46c923a03211e3be3b77c8b83abd6db25b9ca726e89dac5055cbd1d2 not found: ID does not exist" Jul 14 13:29:22.878936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.878902 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:29:22.886172 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:22.886138 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8655ff76dd-xfbcd"] Jul 14 13:29:23.977166 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:23.977135 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:29:23.981425 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:23.981402 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85f8b5cb44-lhjkl" Jul 14 13:29:24.138501 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:24.138472 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" path="/var/lib/kubelet/pods/7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da/volumes" Jul 14 13:29:30.889809 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:30.889776 2568 generic.go:356] "Generic (PLEG): container finished" podID="86639ef1-7e07-495c-a375-1d8a0bf140cf" containerID="3eafa7b497c86565ebc9ffb58960e0edce4553b7c47361caff82283f5907fc49" exitCode=0 Jul 14 13:29:30.890266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:30.889861 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" event={"ID":"86639ef1-7e07-495c-a375-1d8a0bf140cf","Type":"ContainerDied","Data":"3eafa7b497c86565ebc9ffb58960e0edce4553b7c47361caff82283f5907fc49"} Jul 14 13:29:30.890266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:30.890207 2568 scope.go:117] "RemoveContainer" containerID="3eafa7b497c86565ebc9ffb58960e0edce4553b7c47361caff82283f5907fc49" Jul 14 13:29:31.895920 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:31.895890 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-94b28" event={"ID":"86639ef1-7e07-495c-a375-1d8a0bf140cf","Type":"ContainerStarted","Data":"2a5e98971e6bf2653f20ddf90144fb99a65ca1f7b7ab05a6a753108854d5b89a"} Jul 14 13:29:59.596924 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.596891 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:29:59.597323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597181 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="alertmanager" containerID="cri-o://c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" gracePeriod=120 Jul 14 13:29:59.597323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597229 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-metric" containerID="cri-o://faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" gracePeriod=120 Jul 14 13:29:59.597323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597286 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy" containerID="cri-o://eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" gracePeriod=120 Jul 14 13:29:59.597534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597300 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="config-reloader" containerID="cri-o://8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" gracePeriod=120 Jul 14 13:29:59.597534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597317 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="prom-label-proxy" containerID="cri-o://4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" gracePeriod=120 Jul 14 13:29:59.597534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:29:59.597248 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-web" containerID="cri-o://25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" gracePeriod=120 Jul 14 13:30:00.019410 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019318 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" exitCode=0 Jul 14 13:30:00.019410 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019342 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" exitCode=0 Jul 14 13:30:00.019410 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019348 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" exitCode=0 Jul 14 13:30:00.019410 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019354 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" exitCode=0 Jul 14 13:30:00.019410 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019395 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76"} Jul 14 13:30:00.019784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019479 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa"} Jul 14 13:30:00.019784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf"} Jul 14 13:30:00.019784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.019558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f"} Jul 14 13:30:00.845913 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.845884 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:00.980865 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980783 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.980865 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980829 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.980865 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980859 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980904 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980950 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.980978 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981004 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981040 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8blsr\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981077 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981101 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981133 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981130 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981154 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981197 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets\") pod \"6ff6b544-ae62-49e7-bdae-6085428d8d33\" (UID: \"6ff6b544-ae62-49e7-bdae-6085428d8d33\") " Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981258 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981303 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981484 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-metrics-client-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:00.981578 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981507 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-main-db\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:00.981917 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.981598 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:30:00.985078 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.983937 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985395 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985420 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985424 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985448 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume" (OuterVolumeSpecName: "config-volume") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985459 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out" (OuterVolumeSpecName: "config-out") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:30:00.985495 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985488 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.985893 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.985872 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr" (OuterVolumeSpecName: "kube-api-access-8blsr") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "kube-api-access-8blsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:30:00.989244 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.989121 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:00.996113 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:00.996093 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config" (OuterVolumeSpecName: "web-config") pod "6ff6b544-ae62-49e7-bdae-6085428d8d33" (UID: "6ff6b544-ae62-49e7-bdae-6085428d8d33"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:30:01.027277 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027251 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" exitCode=0 Jul 14 13:30:01.027277 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027270 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerID="25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" exitCode=0 Jul 14 13:30:01.027422 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027324 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.027422 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027325 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398"} Jul 14 13:30:01.027422 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027394 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142"} Jul 14 13:30:01.027569 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027425 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6ff6b544-ae62-49e7-bdae-6085428d8d33","Type":"ContainerDied","Data":"2712ed11531b43ee66e9e310374c2721598000bc53c41d0071af10591346325a"} Jul 14 13:30:01.027569 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.027462 2568 scope.go:117] "RemoveContainer" containerID="4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" Jul 14 13:30:01.035547 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.035537 2568 scope.go:117] "RemoveContainer" containerID="faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" Jul 14 13:30:01.042590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.042569 2568 scope.go:117] "RemoveContainer" containerID="eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" Jul 14 13:30:01.049420 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.049401 2568 scope.go:117] "RemoveContainer" containerID="25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" Jul 14 13:30:01.055974 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.055960 2568 scope.go:117] "RemoveContainer" containerID="8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" Jul 14 13:30:01.058392 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.058372 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:30:01.062740 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.062708 2568 scope.go:117] "RemoveContainer" containerID="c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" Jul 14 13:30:01.070662 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.069258 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:30:01.075326 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.075307 2568 scope.go:117] "RemoveContainer" containerID="7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09" Jul 14 13:30:01.081999 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.081979 2568 scope.go:117] "RemoveContainer" containerID="4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" Jul 14 13:30:01.082260 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.082242 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76\": container with ID starting with 4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76 not found: ID does not exist" containerID="4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" Jul 14 13:30:01.082313 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082269 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082313 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082269 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76"} err="failed to get container status \"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76\": rpc error: code = NotFound desc = could not find container \"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76\": container with ID starting with 4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76 not found: ID does not exist" Jul 14 13:30:01.082313 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082285 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-web-config\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082313 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082302 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ff6b544-ae62-49e7-bdae-6085428d8d33-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082313 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082312 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-cluster-tls-config\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082321 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082329 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8blsr\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-kube-api-access-8blsr\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082338 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082347 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-out\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082354 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-config-volume\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082288 2568 scope.go:117] "RemoveContainer" containerID="faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082362 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6ff6b544-ae62-49e7-bdae-6085428d8d33-secret-alertmanager-main-tls\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082497 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082370 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6ff6b544-ae62-49e7-bdae-6085428d8d33-tls-assets\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:30:01.082834 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.082598 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398\": container with ID starting with faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398 not found: ID does not exist" containerID="faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" Jul 14 13:30:01.082834 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082616 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398"} err="failed to get container status \"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398\": rpc error: code = NotFound desc = could not find container \"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398\": container with ID starting with faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398 not found: ID does not exist" Jul 14 13:30:01.082834 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082630 2568 scope.go:117] "RemoveContainer" containerID="eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" Jul 14 13:30:01.082929 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.082849 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa\": container with ID starting with eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa not found: ID does not exist" containerID="eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" Jul 14 13:30:01.082929 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082871 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa"} err="failed to get container status \"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa\": rpc error: code = NotFound desc = could not find container \"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa\": container with ID starting with eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa not found: ID does not exist" Jul 14 13:30:01.082929 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.082884 2568 scope.go:117] "RemoveContainer" containerID="25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" Jul 14 13:30:01.083106 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.083091 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142\": container with ID starting with 25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142 not found: ID does not exist" containerID="25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" Jul 14 13:30:01.083143 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083109 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142"} err="failed to get container status \"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142\": rpc error: code = NotFound desc = could not find container \"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142\": container with ID starting with 25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142 not found: ID does not exist" Jul 14 13:30:01.083143 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083121 2568 scope.go:117] "RemoveContainer" containerID="8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" Jul 14 13:30:01.083353 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.083336 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf\": container with ID starting with 8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf not found: ID does not exist" containerID="8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" Jul 14 13:30:01.083395 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083357 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf"} err="failed to get container status \"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf\": rpc error: code = NotFound desc = could not find container \"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf\": container with ID starting with 8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf not found: ID does not exist" Jul 14 13:30:01.083395 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083371 2568 scope.go:117] "RemoveContainer" containerID="c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" Jul 14 13:30:01.083578 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.083560 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f\": container with ID starting with c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f not found: ID does not exist" containerID="c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" Jul 14 13:30:01.083619 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083581 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f"} err="failed to get container status \"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f\": rpc error: code = NotFound desc = could not find container \"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f\": container with ID starting with c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f not found: ID does not exist" Jul 14 13:30:01.083619 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083592 2568 scope.go:117] "RemoveContainer" containerID="7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09" Jul 14 13:30:01.083856 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:30:01.083839 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09\": container with ID starting with 7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09 not found: ID does not exist" containerID="7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09" Jul 14 13:30:01.083903 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083860 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09"} err="failed to get container status \"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09\": rpc error: code = NotFound desc = could not find container \"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09\": container with ID starting with 7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09 not found: ID does not exist" Jul 14 13:30:01.083903 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.083880 2568 scope.go:117] "RemoveContainer" containerID="4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76" Jul 14 13:30:01.084100 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084084 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76"} err="failed to get container status \"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76\": rpc error: code = NotFound desc = could not find container \"4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76\": container with ID starting with 4608c537d6ba6d29a0239d79dca330393ca74d081ad25235f68807ece31f6b76 not found: ID does not exist" Jul 14 13:30:01.084154 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084101 2568 scope.go:117] "RemoveContainer" containerID="faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398" Jul 14 13:30:01.084316 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084299 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398"} err="failed to get container status \"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398\": rpc error: code = NotFound desc = could not find container \"faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398\": container with ID starting with faa30ff221fdaa9c9714e1730aa6314027e7d047a264a1ecc330cfd463bda398 not found: ID does not exist" Jul 14 13:30:01.084367 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084317 2568 scope.go:117] "RemoveContainer" containerID="eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa" Jul 14 13:30:01.084511 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084496 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa"} err="failed to get container status \"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa\": rpc error: code = NotFound desc = could not find container \"eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa\": container with ID starting with eebc7271ac7f1f68ba3732e3b0747f1f26261ca0b9aa2d7e35817e492ab7f5fa not found: ID does not exist" Jul 14 13:30:01.084548 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084511 2568 scope.go:117] "RemoveContainer" containerID="25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142" Jul 14 13:30:01.084738 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084707 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142"} err="failed to get container status \"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142\": rpc error: code = NotFound desc = could not find container \"25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142\": container with ID starting with 25cd15acdf2f06dbfaf52a610b73c8ff8351fb3cb3986a5c00bb1da236282142 not found: ID does not exist" Jul 14 13:30:01.084786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084744 2568 scope.go:117] "RemoveContainer" containerID="8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf" Jul 14 13:30:01.084957 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084931 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf"} err="failed to get container status \"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf\": rpc error: code = NotFound desc = could not find container \"8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf\": container with ID starting with 8b905c67cf0bb71a369cd009d9139815c2054e02f572640a979c20c153effbcf not found: ID does not exist" Jul 14 13:30:01.084995 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.084957 2568 scope.go:117] "RemoveContainer" containerID="c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f" Jul 14 13:30:01.085152 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.085134 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f"} err="failed to get container status \"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f\": rpc error: code = NotFound desc = could not find container \"c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f\": container with ID starting with c3da61ab324d8bdb710f683571b83817519e42f26cae89a3c8a858349ccdd63f not found: ID does not exist" Jul 14 13:30:01.085196 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.085153 2568 scope.go:117] "RemoveContainer" containerID="7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09" Jul 14 13:30:01.085344 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.085328 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09"} err="failed to get container status \"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09\": rpc error: code = NotFound desc = could not find container \"7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09\": container with ID starting with 7aecced5a3479c7b733dab7b419135c3c9719a6da3e0f067a84b3aa10954ce09 not found: ID does not exist" Jul 14 13:30:01.111051 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111026 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:30:01.111353 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111341 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="init-config-reloader" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111355 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="init-config-reloader" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111364 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" containerName="console" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111369 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" containerName="console" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111376 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="config-reloader" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111381 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="config-reloader" Jul 14 13:30:01.111394 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111393 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="alertmanager" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111399 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="alertmanager" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111410 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-web" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111415 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-web" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111422 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="prom-label-proxy" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111428 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="prom-label-proxy" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111436 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-metric" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111442 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-metric" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111448 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111453 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111460 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee0bc73-1865-422b-8b11-726b3d376674" containerName="registry" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111465 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee0bc73-1865-422b-8b11-726b3d376674" containerName="registry" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111514 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-metric" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111521 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2f2d25-0ff3-4773-a9d7-8dcb495bc7da" containerName="console" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111527 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee0bc73-1865-422b-8b11-726b3d376674" containerName="registry" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111532 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="alertmanager" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111537 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="config-reloader" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111544 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="prom-label-proxy" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111549 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy-web" Jul 14 13:30:01.111574 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.111554 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" containerName="kube-rbac-proxy" Jul 14 13:30:01.115467 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.115452 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.119139 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.119120 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5ssn4\"" Jul 14 13:30:01.119262 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.119247 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 13:30:01.119325 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.119280 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 13:30:01.119585 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.119570 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 13:30:01.120390 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.120371 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 13:30:01.120499 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.120392 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 13:30:01.120499 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.120404 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 13:30:01.121056 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.121042 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 13:30:01.121677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.121657 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 13:30:01.128003 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.127986 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 13:30:01.136914 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.136891 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:30:01.183295 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183261 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183295 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183430 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183461 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183701 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183701 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdm45\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-kube-api-access-zdm45\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183701 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-config-out\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183877 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183877 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183750 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.183877 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-web-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.184002 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.183876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.284770 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdm45\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-kube-api-access-zdm45\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284849 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-config-out\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284883 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284916 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285048 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.284949 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-web-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.285070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.285122 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.285152 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.285161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.285377 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.285198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.286224 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.286194 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.286340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.286256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.286340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.286288 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.287852 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.287717 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d1c1411-2425-4548-996e-60b2a89b10e8-config-out\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.287954 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.287939 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288018 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-web-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288135 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288175 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288147 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288175 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288149 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-config-volume\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288521 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d1c1411-2425-4548-996e-60b2a89b10e8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288647 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.288830 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.288811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.289814 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.289799 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9d1c1411-2425-4548-996e-60b2a89b10e8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.294779 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.294758 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdm45\" (UniqueName: \"kubernetes.io/projected/9d1c1411-2425-4548-996e-60b2a89b10e8-kube-api-access-zdm45\") pod \"alertmanager-main-0\" (UID: \"9d1c1411-2425-4548-996e-60b2a89b10e8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.425223 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.425183 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:30:01.572841 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:01.572651 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:30:01.575557 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:30:01.575530 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1c1411_2425_4548_996e_60b2a89b10e8.slice/crio-d4bb203893bc184d8b2ba80e9c40569226f75a2b1e89c009891fbea4c66b8330 WatchSource:0}: Error finding container d4bb203893bc184d8b2ba80e9c40569226f75a2b1e89c009891fbea4c66b8330: Status 404 returned error can't find the container with id d4bb203893bc184d8b2ba80e9c40569226f75a2b1e89c009891fbea4c66b8330 Jul 14 13:30:02.034400 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:02.034364 2568 generic.go:356] "Generic (PLEG): container finished" podID="9d1c1411-2425-4548-996e-60b2a89b10e8" containerID="311d018607f168ba953200c653774de6b79ace1b76c1e227cb5b08f114db1763" exitCode=0 Jul 14 13:30:02.034899 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:02.034411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerDied","Data":"311d018607f168ba953200c653774de6b79ace1b76c1e227cb5b08f114db1763"} Jul 14 13:30:02.034899 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:02.034465 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"d4bb203893bc184d8b2ba80e9c40569226f75a2b1e89c009891fbea4c66b8330"} Jul 14 13:30:02.139157 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:02.139129 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ff6b544-ae62-49e7-bdae-6085428d8d33" path="/var/lib/kubelet/pods/6ff6b544-ae62-49e7-bdae-6085428d8d33/volumes" Jul 14 13:30:03.042441 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042404 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"6f2e259417a9a3b82aa1aeafc846a31e896124c850cb8463f19ee43ed6492c4b"} Jul 14 13:30:03.042836 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"6da892f875f5f24d0ccbefc0eff31924c4781080392473de7fd2bd4d2e3a2020"} Jul 14 13:30:03.042836 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042496 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"d8154b16a545ac79c3731f3f4834bd0fc59a31119eddd6b6d3debf35fa412a6a"} Jul 14 13:30:03.042836 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"1d5ae76e0a944cbce94f9904852707a2520176f84b51d343f25fe99ddfcf945f"} Jul 14 13:30:03.042836 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"91e6eea603fbd63f039cc117a066a4b38511fbde94ffcdbebb54606a475b1151"} Jul 14 13:30:03.042836 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.042573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9d1c1411-2425-4548-996e-60b2a89b10e8","Type":"ContainerStarted","Data":"86e0a830c9545fcc194e4b55c15f5b7ed67f214d3a368dd35ae22d8ec0651684"} Jul 14 13:30:03.080417 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:30:03.080350 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.06368816 podStartE2EDuration="2.080325688s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.637528ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:30:03.077973888 +0000 UTC m=+169.544101467" watchObservedRunningTime="2026-07-14 13:30:03.080325688 +0000 UTC m=+169.546453242" Jul 14 13:31:35.805791 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.805695 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-987hq"] Jul 14 13:31:35.808288 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.808267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.811110 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.811092 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 13:31:35.812459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.812438 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 13:31:35.812459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.812448 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bd7t8\"" Jul 14 13:31:35.818616 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.818593 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-987hq"] Jul 14 13:31:35.887632 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.887590 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p59k\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-kube-api-access-4p59k\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.887844 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.887645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.988329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.988289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p59k\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-kube-api-access-4p59k\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.988329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.988338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.998409 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.998372 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:35.998537 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:35.998424 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p59k\" (UniqueName: \"kubernetes.io/projected/5fa9f4c2-346b-41b9-a1f1-100bddf18c1b-kube-api-access-4p59k\") pod \"cert-manager-webhook-6486ffd997-987hq\" (UID: \"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:36.123954 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:36.123865 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:36.261530 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:36.261497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-987hq"] Jul 14 13:31:36.264952 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:31:36.264917 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fa9f4c2_346b_41b9_a1f1_100bddf18c1b.slice/crio-c19dd91ae136a416dbf4d81894dc3987b39a87536e741704f82e79183a326b98 WatchSource:0}: Error finding container c19dd91ae136a416dbf4d81894dc3987b39a87536e741704f82e79183a326b98: Status 404 returned error can't find the container with id c19dd91ae136a416dbf4d81894dc3987b39a87536e741704f82e79183a326b98 Jul 14 13:31:36.431721 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:36.431634 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" event={"ID":"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b","Type":"ContainerStarted","Data":"c19dd91ae136a416dbf4d81894dc3987b39a87536e741704f82e79183a326b98"} Jul 14 13:31:39.447740 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:39.447704 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" event={"ID":"5fa9f4c2-346b-41b9-a1f1-100bddf18c1b","Type":"ContainerStarted","Data":"9ff5ef5ffe02298a1d8f3e5c2bc5408b2f06441157bfdaecccece55a6f7022d2"} Jul 14 13:31:39.448147 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:39.447872 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:39.468675 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:39.468625 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" podStartSLOduration=1.42252379 podStartE2EDuration="4.468610084s" totalImagesPullingTime="3.046086294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:31:39.467549016 +0000 UTC m=+265.933676571" watchObservedRunningTime="2026-07-14 13:31:39.468610084 +0000 UTC m=+265.934737637" Jul 14 13:31:42.287913 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.287875 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7"] Jul 14 13:31:42.291931 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.291900 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.295323 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.295288 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 13:31:42.295468 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.295411 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:31:42.296799 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.296785 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bhtmg\"" Jul 14 13:31:42.301164 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.301140 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7"] Jul 14 13:31:42.347685 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.347651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aafaee6-5695-44ad-94b8-02025064d64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.347685 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.347688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6pmc\" (UniqueName: \"kubernetes.io/projected/7aafaee6-5695-44ad-94b8-02025064d64a-kube-api-access-l6pmc\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.448457 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.448407 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aafaee6-5695-44ad-94b8-02025064d64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.448457 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.448462 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6pmc\" (UniqueName: \"kubernetes.io/projected/7aafaee6-5695-44ad-94b8-02025064d64a-kube-api-access-l6pmc\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.448833 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.448811 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7aafaee6-5695-44ad-94b8-02025064d64a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.459025 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.458996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6pmc\" (UniqueName: \"kubernetes.io/projected/7aafaee6-5695-44ad-94b8-02025064d64a-kube-api-access-l6pmc\") pod \"openshift-lws-operator-fd8ccff4c-b8lg7\" (UID: \"7aafaee6-5695-44ad-94b8-02025064d64a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.601769 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.601658 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" Jul 14 13:31:42.745361 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:42.745322 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7"] Jul 14 13:31:42.749223 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:31:42.749193 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aafaee6_5695_44ad_94b8_02025064d64a.slice/crio-ab9ed4f17ded82db664eeffad41f1cdcaed5e2bee7e33a6ebf67aa9d67653cdd WatchSource:0}: Error finding container ab9ed4f17ded82db664eeffad41f1cdcaed5e2bee7e33a6ebf67aa9d67653cdd: Status 404 returned error can't find the container with id ab9ed4f17ded82db664eeffad41f1cdcaed5e2bee7e33a6ebf67aa9d67653cdd Jul 14 13:31:43.465741 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:43.465684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" event={"ID":"7aafaee6-5695-44ad-94b8-02025064d64a","Type":"ContainerStarted","Data":"ab9ed4f17ded82db664eeffad41f1cdcaed5e2bee7e33a6ebf67aa9d67653cdd"} Jul 14 13:31:45.453777 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:45.453724 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-987hq" Jul 14 13:31:45.477668 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:45.477631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" event={"ID":"7aafaee6-5695-44ad-94b8-02025064d64a","Type":"ContainerStarted","Data":"36e67c12c65a092bd164763d0cbc9055e7619867d2b57343a96888790d9e1743"} Jul 14 13:31:45.499533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:31:45.499482 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-b8lg7" podStartSLOduration=0.964502147 podStartE2EDuration="3.499461337s" totalImagesPullingTime="2.53495919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:31:45.498485713 +0000 UTC m=+271.964613268" watchObservedRunningTime="2026-07-14 13:31:45.499461337 +0000 UTC m=+271.965588891" Jul 14 13:32:01.440333 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.440303 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps"] Jul 14 13:32:01.443070 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.443048 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.447504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.447475 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 13:32:01.447504 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.447494 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 13:32:01.447671 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.447480 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 13:32:01.448245 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.448220 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 13:32:01.449505 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.449486 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b48w9\"" Jul 14 13:32:01.469516 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.469481 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps"] Jul 14 13:32:01.517255 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.517225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.517255 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.517258 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.517459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.517287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.517459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.517357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46jzw\" (UniqueName: \"kubernetes.io/projected/b654cbd9-ba91-400b-9d79-13fbd6f324e6-kube-api-access-46jzw\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.517459 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.517393 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618189 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618147 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618189 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618187 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618211 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46jzw\" (UniqueName: \"kubernetes.io/projected/b654cbd9-ba91-400b-9d79-13fbd6f324e6-kube-api-access-46jzw\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.618662 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.618619 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-manifests\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.620446 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.620421 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b654cbd9-ba91-400b-9d79-13fbd6f324e6-tmp\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.620765 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.620744 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.620821 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.620803 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b654cbd9-ba91-400b-9d79-13fbd6f324e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.635773 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.635738 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46jzw\" (UniqueName: \"kubernetes.io/projected/b654cbd9-ba91-400b-9d79-13fbd6f324e6-kube-api-access-46jzw\") pod \"opendatahub-operator-controller-manager-9464f545f-64kps\" (UID: \"b654cbd9-ba91-400b-9d79-13fbd6f324e6\") " pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.754037 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.753955 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:01.929988 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:01.929959 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps"] Jul 14 13:32:01.931430 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:32:01.931402 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb654cbd9_ba91_400b_9d79_13fbd6f324e6.slice/crio-1a8f81fb2530686b74076a9fb8100c2c0328cbdccb7bf538874aefd5b3ce515f WatchSource:0}: Error finding container 1a8f81fb2530686b74076a9fb8100c2c0328cbdccb7bf538874aefd5b3ce515f: Status 404 returned error can't find the container with id 1a8f81fb2530686b74076a9fb8100c2c0328cbdccb7bf538874aefd5b3ce515f Jul 14 13:32:02.556892 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:02.556858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" event={"ID":"b654cbd9-ba91-400b-9d79-13fbd6f324e6","Type":"ContainerStarted","Data":"1a8f81fb2530686b74076a9fb8100c2c0328cbdccb7bf538874aefd5b3ce515f"} Jul 14 13:32:05.573910 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:05.573872 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" event={"ID":"b654cbd9-ba91-400b-9d79-13fbd6f324e6","Type":"ContainerStarted","Data":"0ed1f5139de02a796ca70929079ece45dbc22ca521b5ac04ecae8c7e85a66cee"} Jul 14 13:32:06.580363 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:06.580332 2568 generic.go:356] "Generic (PLEG): container finished" podID="b654cbd9-ba91-400b-9d79-13fbd6f324e6" containerID="0ed1f5139de02a796ca70929079ece45dbc22ca521b5ac04ecae8c7e85a66cee" exitCode=0 Jul 14 13:32:06.580762 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:06.580369 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" event={"ID":"b654cbd9-ba91-400b-9d79-13fbd6f324e6","Type":"ContainerDied","Data":"0ed1f5139de02a796ca70929079ece45dbc22ca521b5ac04ecae8c7e85a66cee"} Jul 14 13:32:07.586754 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:07.586703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" event={"ID":"b654cbd9-ba91-400b-9d79-13fbd6f324e6","Type":"ContainerStarted","Data":"b050469d8b91a7176e3dd6601a84a2aaceffeb5306a71deb5373cbe19aa87dde"} Jul 14 13:32:07.587167 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:07.586779 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:07.673552 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:07.673496 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" podStartSLOduration=2.16875149 podStartE2EDuration="6.67347635s" totalImagesPullingTime="3.483761937s" totalInitContainerRuntime="1.020962923s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:07.671095102 +0000 UTC m=+294.137222669" watchObservedRunningTime="2026-07-14 13:32:07.67347635 +0000 UTC m=+294.139603903" Jul 14 13:32:13.969229 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:13.969193 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:32:13.971401 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:13.971374 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:32:13.976938 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:13.976910 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:32:13.985205 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:13.985176 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:32:13.987494 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:13.987476 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 13:32:18.593489 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:18.593463 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9464f545f-64kps" Jul 14 13:32:21.288776 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.288740 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd"] Jul 14 13:32:21.293167 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.293147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.296258 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.296238 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 13:32:21.296366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.296275 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 13:32:21.297483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.297465 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7zflm\"" Jul 14 13:32:21.297554 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.297538 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 13:32:21.303105 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.303085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhrjr\" (UniqueName: \"kubernetes.io/projected/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-kube-api-access-lhrjr\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.303209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.303129 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-manager-config\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.303209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.303149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.303327 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.303302 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-metrics-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.307367 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.307348 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd"] Jul 14 13:32:21.403778 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.403718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-manager-config\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.403778 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.403776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.404025 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.403843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-metrics-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.404025 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.403970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhrjr\" (UniqueName: \"kubernetes.io/projected/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-kube-api-access-lhrjr\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.404436 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.404406 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-manager-config\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.406343 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.406323 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-metrics-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.406437 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.406393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-cert\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.425611 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.425578 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhrjr\" (UniqueName: \"kubernetes.io/projected/f3352af1-a3ad-42e8-8fc8-3f1b98a082b1-kube-api-access-lhrjr\") pod \"lws-controller-manager-fc4b75864-gq4sd\" (UID: \"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1\") " pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.602919 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.602820 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:21.823688 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.823631 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd"] Jul 14 13:32:21.826520 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:32:21.826493 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3352af1_a3ad_42e8_8fc8_3f1b98a082b1.slice/crio-7f14ddbb8b2c8db63db8e3b038cf53a3dffc70ec432bf1438840a2cf32e6b0ba WatchSource:0}: Error finding container 7f14ddbb8b2c8db63db8e3b038cf53a3dffc70ec432bf1438840a2cf32e6b0ba: Status 404 returned error can't find the container with id 7f14ddbb8b2c8db63db8e3b038cf53a3dffc70ec432bf1438840a2cf32e6b0ba Jul 14 13:32:21.828442 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:21.828428 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:32:22.659855 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:22.659817 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" event={"ID":"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1","Type":"ContainerStarted","Data":"7f14ddbb8b2c8db63db8e3b038cf53a3dffc70ec432bf1438840a2cf32e6b0ba"} Jul 14 13:32:23.665322 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:23.665291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" event={"ID":"f3352af1-a3ad-42e8-8fc8-3f1b98a082b1","Type":"ContainerStarted","Data":"aba7d7eabf539da7709cedd976b9f3a0c1254ed737dd2114de0744d310f468b4"} Jul 14 13:32:23.665794 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:23.665408 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:32:23.715584 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:23.715530 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" podStartSLOduration=1.4152237269999999 podStartE2EDuration="2.715514164s" totalImagesPullingTime="1.300290437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:23.714342424 +0000 UTC m=+310.180469998" watchObservedRunningTime="2026-07-14 13:32:23.715514164 +0000 UTC m=+310.181641717" Jul 14 13:32:29.381772 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.381719 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45"] Jul 14 13:32:29.385545 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.385521 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.388367 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.388345 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 13:32:29.388483 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.388373 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rxxx8\"" Jul 14 13:32:29.388803 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.388788 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 13:32:29.389020 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.389002 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 13:32:29.389234 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.389221 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 13:32:29.398513 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.398490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45"] Jul 14 13:32:29.472524 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472524 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7klx\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-kube-api-access-v7klx\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43167fe-b827-4bce-b62a-e991cdd27eb1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.472786 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.472757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573300 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573351 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7klx\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-kube-api-access-v7klx\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43167fe-b827-4bce-b62a-e991cdd27eb1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573404 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.573830 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.573548 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.574104 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.574082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.575974 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.575945 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e43167fe-b827-4bce-b62a-e991cdd27eb1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.576155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.576134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.576489 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.576473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.576647 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.576625 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.584296 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.584266 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7klx\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-kube-api-access-v7klx\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.584407 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.584391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e43167fe-b827-4bce-b62a-e991cdd27eb1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ndh45\" (UID: \"e43167fe-b827-4bce-b62a-e991cdd27eb1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.695202 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.695118 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:29.850841 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:29.850813 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45"] Jul 14 13:32:29.853298 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:32:29.853276 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43167fe_b827_4bce_b62a_e991cdd27eb1.slice/crio-0c2dd6e1cc21bc279b1ce9fde210e31f9ea7c6671a0d06aec510f120ad633d0e WatchSource:0}: Error finding container 0c2dd6e1cc21bc279b1ce9fde210e31f9ea7c6671a0d06aec510f120ad633d0e: Status 404 returned error can't find the container with id 0c2dd6e1cc21bc279b1ce9fde210e31f9ea7c6671a0d06aec510f120ad633d0e Jul 14 13:32:30.697946 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:30.697905 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" event={"ID":"e43167fe-b827-4bce-b62a-e991cdd27eb1","Type":"ContainerStarted","Data":"0c2dd6e1cc21bc279b1ce9fde210e31f9ea7c6671a0d06aec510f120ad633d0e"} Jul 14 13:32:32.611890 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:32.611844 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 13:32:32.612175 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:32.611920 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 13:32:32.712925 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:32.712649 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" event={"ID":"e43167fe-b827-4bce-b62a-e991cdd27eb1","Type":"ContainerStarted","Data":"743df6113c01abb6fdefc69f6c5d5d95b456f2bd55df2a8520abeb836b9d9f35"} Jul 14 13:32:32.713126 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:32.712939 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:32.755820 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:32.755763 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" podStartSLOduration=0.999757772 podStartE2EDuration="3.755723247s" totalImagesPullingTime="2.755965475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:32:32.754978811 +0000 UTC m=+319.221106370" watchObservedRunningTime="2026-07-14 13:32:32.755723247 +0000 UTC m=+319.221850800" Jul 14 13:32:33.719058 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:33.719023 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ndh45" Jul 14 13:32:34.671688 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:32:34.671662 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-fc4b75864-gq4sd" Jul 14 13:34:25.107233 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.107200 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj"] Jul 14 13:34:25.110520 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.110506 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.132585 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.132556 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 13:34:25.132585 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.132567 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xnwll\"" Jul 14 13:34:25.132779 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.132659 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 13:34:25.146805 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.146775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.146921 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.146898 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2c7t\" (UniqueName: \"kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.157763 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.157724 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj"] Jul 14 13:34:25.247867 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.247829 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.248070 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.247948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2c7t\" (UniqueName: \"kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.248209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.248189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.267251 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.267219 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2c7t\" (UniqueName: \"kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.420132 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.420033 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:25.566878 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.566843 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj"] Jul 14 13:34:25.570971 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:34:25.570941 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688ab960_9cc2_43d9_9544_5c70fe7bf76c.slice/crio-95df620a79cdd11a5b31746cfacb3f3e4d01279f60e9596712460c4724d6995e WatchSource:0}: Error finding container 95df620a79cdd11a5b31746cfacb3f3e4d01279f60e9596712460c4724d6995e: Status 404 returned error can't find the container with id 95df620a79cdd11a5b31746cfacb3f3e4d01279f60e9596712460c4724d6995e Jul 14 13:34:25.575768 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.575741 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj"] Jul 14 13:34:25.582705 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.582681 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj"] Jul 14 13:34:25.609462 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.609432 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:25.614051 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.614028 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.627531 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.627507 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:25.650702 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.650669 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.650926 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.650770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrfd6\" (UniqueName: \"kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.752127 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.752027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.752280 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.752120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrfd6\" (UniqueName: \"kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.752435 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.752406 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.783800 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.783767 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrfd6\" (UniqueName: \"kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6\") pod \"kuadrant-operator-controller-manager-55467d9f7-zdjdh\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:25.924167 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:25.924134 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:26.071371 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:26.071345 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:26.074214 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:34:26.074189 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1bb15b_dd01_4d3e_949a_6030db3bd38c.slice/crio-4f26c038787f0cce790e9d5c94c8cc4758f5c96a5e919855b72f860f628a76f7 WatchSource:0}: Error finding container 4f26c038787f0cce790e9d5c94c8cc4758f5c96a5e919855b72f860f628a76f7: Status 404 returned error can't find the container with id 4f26c038787f0cce790e9d5c94c8cc4758f5c96a5e919855b72f860f628a76f7 Jul 14 13:34:26.252804 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:26.252767 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" event={"ID":"9c1bb15b-dd01-4d3e-949a-6030db3bd38c","Type":"ContainerStarted","Data":"4f26c038787f0cce790e9d5c94c8cc4758f5c96a5e919855b72f860f628a76f7"} Jul 14 13:34:31.287378 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.287342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" event={"ID":"9c1bb15b-dd01-4d3e-949a-6030db3bd38c","Type":"ContainerStarted","Data":"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1"} Jul 14 13:34:31.287821 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.287411 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:31.288886 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.288862 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" containerName="manager" containerID="cri-o://e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68" gracePeriod=2 Jul 14 13:34:31.315218 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.315158 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" podStartSLOduration=1.540609788 podStartE2EDuration="6.315137339s" totalImagesPullingTime="4.774527551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:34:31.313073745 +0000 UTC m=+437.779201319" watchObservedRunningTime="2026-07-14 13:34:31.315137339 +0000 UTC m=+437.781264893" Jul 14 13:34:31.528162 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.528139 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:31.530792 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:34:31.530757 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:31.601427 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.601347 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume\") pod \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " Jul 14 13:34:31.601427 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.601382 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2c7t\" (UniqueName: \"kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t\") pod \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\" (UID: \"688ab960-9cc2-43d9-9544-5c70fe7bf76c\") " Jul 14 13:34:31.601651 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.601631 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "688ab960-9cc2-43d9-9544-5c70fe7bf76c" (UID: "688ab960-9cc2-43d9-9544-5c70fe7bf76c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:34:31.603588 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.603561 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t" (OuterVolumeSpecName: "kube-api-access-c2c7t") pod "688ab960-9cc2-43d9-9544-5c70fe7bf76c" (UID: "688ab960-9cc2-43d9-9544-5c70fe7bf76c"). InnerVolumeSpecName "kube-api-access-c2c7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:34:31.702198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.702164 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/688ab960-9cc2-43d9-9544-5c70fe7bf76c-extensions-socket-volume\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:34:31.702198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:31.702192 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2c7t\" (UniqueName: \"kubernetes.io/projected/688ab960-9cc2-43d9-9544-5c70fe7bf76c-kube-api-access-c2c7t\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:34:32.138045 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.137965 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" path="/var/lib/kubelet/pods/688ab960-9cc2-43d9-9544-5c70fe7bf76c/volumes" Jul 14 13:34:32.300255 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.300218 2568 generic.go:356] "Generic (PLEG): container finished" podID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" containerID="e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68" exitCode=2 Jul 14 13:34:32.300255 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.300259 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:32.300785 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.300314 2568 scope.go:117] "RemoveContainer" containerID="e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68" Jul 14 13:34:32.305446 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:34:32.305419 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:32.308648 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.308632 2568 scope.go:117] "RemoveContainer" containerID="e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68" Jul 14 13:34:32.308932 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:34:32.308914 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68\": container with ID starting with e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68 not found: ID does not exist" containerID="e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68" Jul 14 13:34:32.308988 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:32.308939 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68"} err="failed to get container status \"e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68\": rpc error: code = NotFound desc = could not find container \"e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68\": container with ID starting with e77ea1c47e10dab854d97d67bcfe404956416bc9accd7f3596bd999a20adbd68 not found: ID does not exist" Jul 14 13:34:34.136646 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:34:34.136608 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-9qdfj\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9qdfj" Jul 14 13:34:42.302989 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:42.302914 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:54.529606 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.529572 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:54.530050 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.529811 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" podUID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" containerName="manager" containerID="cri-o://c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1" gracePeriod=10 Jul 14 13:34:54.783069 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.783013 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:54.899915 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.899881 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume\") pod \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " Jul 14 13:34:54.900116 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.899950 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrfd6\" (UniqueName: \"kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6\") pod \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\" (UID: \"9c1bb15b-dd01-4d3e-949a-6030db3bd38c\") " Jul 14 13:34:54.900411 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.900388 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9c1bb15b-dd01-4d3e-949a-6030db3bd38c" (UID: "9c1bb15b-dd01-4d3e-949a-6030db3bd38c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:34:54.902020 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:54.901997 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6" (OuterVolumeSpecName: "kube-api-access-lrfd6") pod "9c1bb15b-dd01-4d3e-949a-6030db3bd38c" (UID: "9c1bb15b-dd01-4d3e-949a-6030db3bd38c"). InnerVolumeSpecName "kube-api-access-lrfd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:34:55.001029 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.000999 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-extensions-socket-volume\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:34:55.001029 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.001023 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrfd6\" (UniqueName: \"kubernetes.io/projected/9c1bb15b-dd01-4d3e-949a-6030db3bd38c-kube-api-access-lrfd6\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:34:55.409110 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.409076 2568 generic.go:356] "Generic (PLEG): container finished" podID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" containerID="c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1" exitCode=0 Jul 14 13:34:55.409331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.409126 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" Jul 14 13:34:55.409331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.409139 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" event={"ID":"9c1bb15b-dd01-4d3e-949a-6030db3bd38c","Type":"ContainerDied","Data":"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1"} Jul 14 13:34:55.409331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.409181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh" event={"ID":"9c1bb15b-dd01-4d3e-949a-6030db3bd38c","Type":"ContainerDied","Data":"4f26c038787f0cce790e9d5c94c8cc4758f5c96a5e919855b72f860f628a76f7"} Jul 14 13:34:55.409331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.409203 2568 scope.go:117] "RemoveContainer" containerID="c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1" Jul 14 13:34:55.418302 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.418287 2568 scope.go:117] "RemoveContainer" containerID="c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1" Jul 14 13:34:55.418578 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:34:55.418561 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1\": container with ID starting with c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1 not found: ID does not exist" containerID="c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1" Jul 14 13:34:55.418635 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.418585 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1"} err="failed to get container status \"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1\": rpc error: code = NotFound desc = could not find container \"c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1\": container with ID starting with c1cb437568cd6e520f56b7e2a5dc0d7e3a2eef24778f085dafb9758b16dd50d1 not found: ID does not exist" Jul 14 13:34:55.435208 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.435186 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:55.441791 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:55.441769 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zdjdh"] Jul 14 13:34:56.137213 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:34:56.137184 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" path="/var/lib/kubelet/pods/9c1bb15b-dd01-4d3e-949a-6030db3bd38c/volumes" Jul 14 13:35:13.826660 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.826623 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.826977 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" containerName="manager" Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.826988 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" containerName="manager" Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.827016 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" containerName="manager" Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.827022 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" containerName="manager" Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.827076 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c1bb15b-dd01-4d3e-949a-6030db3bd38c" containerName="manager" Jul 14 13:35:13.827155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.827084 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="688ab960-9cc2-43d9-9544-5c70fe7bf76c" containerName="manager" Jul 14 13:35:13.829215 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.829187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:13.833071 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.833051 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fspqb\"" Jul 14 13:35:13.833224 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.833074 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 13:35:13.834329 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.834312 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 13:35:13.834439 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.834349 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 13:35:13.843887 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.843860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:35:13.854781 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.854757 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:35:13.975684 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.975656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90a78eef-508c-4e3d-a745-e69dc89dbda1-config-file\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:13.975886 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:13.975782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r667c\" (UniqueName: \"kubernetes.io/projected/90a78eef-508c-4e3d-a745-e69dc89dbda1-kube-api-access-r667c\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.077198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.077109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r667c\" (UniqueName: \"kubernetes.io/projected/90a78eef-508c-4e3d-a745-e69dc89dbda1-kube-api-access-r667c\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.077198 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.077188 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90a78eef-508c-4e3d-a745-e69dc89dbda1-config-file\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.080396 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.080376 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 14 13:35:14.087818 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.087793 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/90a78eef-508c-4e3d-a745-e69dc89dbda1-config-file\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.088576 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.088560 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 13:35:14.099624 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.099598 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 13:35:14.109044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.109015 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r667c\" (UniqueName: \"kubernetes.io/projected/90a78eef-508c-4e3d-a745-e69dc89dbda1-kube-api-access-r667c\") pod \"limitador-limitador-54d6d988d5-76nk5\" (UID: \"90a78eef-508c-4e3d-a745-e69dc89dbda1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.144093 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.144067 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fspqb\"" Jul 14 13:35:14.150635 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.150601 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:14.293556 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.293527 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:35:14.298485 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:35:14.298448 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a78eef_508c_4e3d_a745_e69dc89dbda1.slice/crio-28e0fb43e81bb3090176535b337d5fb79bde166b37c99a8ecc8124a80a022dcf WatchSource:0}: Error finding container 28e0fb43e81bb3090176535b337d5fb79bde166b37c99a8ecc8124a80a022dcf: Status 404 returned error can't find the container with id 28e0fb43e81bb3090176535b337d5fb79bde166b37c99a8ecc8124a80a022dcf Jul 14 13:35:14.499351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:14.499262 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" event={"ID":"90a78eef-508c-4e3d-a745-e69dc89dbda1","Type":"ContainerStarted","Data":"28e0fb43e81bb3090176535b337d5fb79bde166b37c99a8ecc8124a80a022dcf"} Jul 14 13:35:17.516347 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:17.516312 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" event={"ID":"90a78eef-508c-4e3d-a745-e69dc89dbda1","Type":"ContainerStarted","Data":"bb5b3c1909d4405ce27baf16f67ab2efc97e95780ac55c36a9638cfba83d6ef3"} Jul 14 13:35:17.516776 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:17.516372 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:35:17.538297 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:17.538254 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" podStartSLOduration=1.912125058 podStartE2EDuration="4.538240079s" totalImagesPullingTime="2.626115021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:35:17.536869601 +0000 UTC m=+484.002997206" watchObservedRunningTime="2026-07-14 13:35:17.538240079 +0000 UTC m=+484.004367649" Jul 14 13:35:28.522046 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:35:28.522016 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-76nk5" Jul 14 13:36:31.014191 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:36:31.014152 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:36:49.581535 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:36:49.581497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:36:56.300084 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:36:56.300039 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:37:14.008633 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:14.008598 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:37:14.009203 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:14.008858 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:37:14.014319 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:14.014299 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:37:14.014818 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:14.014796 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:37:15.704985 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:15.704950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:37:18.225442 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.225409 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x"] Jul 14 13:37:18.228738 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.228709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.232152 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.232134 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mmf2v\"" Jul 14 13:37:18.232250 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.232137 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 13:37:18.232344 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.232270 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 13:37:18.235792 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.235776 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 13:37:18.250552 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.250527 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x"] Jul 14 13:37:18.271295 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.271295 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.271501 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271318 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjvxp\" (UniqueName: \"kubernetes.io/projected/d0455cd9-40de-4079-a783-f2617faeb385-kube-api-access-hjvxp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.271501 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0455cd9-40de-4079-a783-f2617faeb385-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.271501 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.271826 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.271502 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372798 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372841 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjvxp\" (UniqueName: \"kubernetes.io/projected/d0455cd9-40de-4079-a783-f2617faeb385-kube-api-access-hjvxp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372922 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0455cd9-40de-4079-a783-f2617faeb385-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.372977 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.372957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.373242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.373220 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.373303 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.373259 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.373351 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.373299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.375146 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.375127 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0455cd9-40de-4079-a783-f2617faeb385-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.375407 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.375390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0455cd9-40de-4079-a783-f2617faeb385-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.385859 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.385840 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjvxp\" (UniqueName: \"kubernetes.io/projected/d0455cd9-40de-4079-a783-f2617faeb385-kube-api-access-hjvxp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x\" (UID: \"d0455cd9-40de-4079-a783-f2617faeb385\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.539065 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.539038 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:18.678577 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:18.678554 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x"] Jul 14 13:37:19.092653 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:19.092614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" event={"ID":"d0455cd9-40de-4079-a783-f2617faeb385","Type":"ContainerStarted","Data":"f258e147a0fc460712aa78e0b28dc3cffca9ed9758c188cf5341eb4793e7d884"} Jul 14 13:37:20.099021 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:20.098984 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" event={"ID":"d0455cd9-40de-4079-a783-f2617faeb385","Type":"ContainerStarted","Data":"c4df63c94cfb2f3baaee9c95893c41af6039a7d6b5d706b8af841d5ad0084c56"} Jul 14 13:37:20.128004 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:20.127946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" podStartSLOduration=1.109822417 podStartE2EDuration="2.127925968s" totalImagesPullingTime="1.018103551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:37:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:37:20.126531941 +0000 UTC m=+606.592659495" watchObservedRunningTime="2026-07-14 13:37:20.127925968 +0000 UTC m=+606.594053523" Jul 14 13:37:20.312216 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:20.312173 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:37:28.540155 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:28.540108 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:28.540566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:28.540178 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:28.552924 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:28.552901 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:29.152005 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:29.151971 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x" Jul 14 13:37:32.105525 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:32.105490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:37:53.415722 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:37:53.415628 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:39:32.801012 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:39:32.800976 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:39:43.995712 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:39:43.995679 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:39:53.799814 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:39:53.799776 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:40:03.388111 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:40:03.388070 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:40:14.593362 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:40:14.593322 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:40:42.196677 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:40:42.196591 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:40:54.199044 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:40:54.199008 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:41:32.200128 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:41:32.200089 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:41:43.293594 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:41:43.293564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:41:52.410215 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:41:52.410182 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:42:03.490703 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:03.490670 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:42:13.307151 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:13.307072 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:42:14.042045 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:14.042019 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:42:14.042231 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:14.042019 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:42:14.047397 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:14.047369 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:42:14.047642 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:14.047628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:42:30.497582 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:30.497547 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:42:39.599613 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:39.599576 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:42:48.992978 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:42:48.992933 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:43:00.001242 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:43:00.001206 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:43:16.099816 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:43:16.099774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:43:27.904615 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:43:27.904577 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:43:52.826263 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:43:52.826169 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:44:03.288533 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:44:03.288498 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:44:19.927403 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:44:19.927366 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:44:29.294316 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:44:29.294281 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:44:40.103604 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:44:40.103570 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:44:50.203867 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:44:50.203835 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:45:00.188419 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.188387 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt"] Jul 14 13:45:00.190801 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.190785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" Jul 14 13:45:00.194482 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.194458 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:45:00.200230 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.200211 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" Jul 14 13:45:00.256243 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.256208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt"] Jul 14 13:45:00.543401 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.538137 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt"] Jul 14 13:45:00.545614 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:45:00.545577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40b268c_02f6_422c_9b66_93c842f35095.slice/crio-c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023 WatchSource:0}: Error finding container c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023: Status 404 returned error can't find the container with id c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023 Jul 14 13:45:00.547332 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.547296 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:45:00.996031 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:00.995992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:45:01.311839 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:01.311799 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" event={"ID":"b40b268c-02f6-422c-9b66-93c842f35095","Type":"ContainerStarted","Data":"c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023"} Jul 14 13:45:04.330672 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:04.330634 2568 generic.go:356] "Generic (PLEG): container finished" podID="b40b268c-02f6-422c-9b66-93c842f35095" containerID="2998898461ec70511b12df82f4e1ee7a6ccf21039a4163b634592725e2fe474f" exitCode=0 Jul 14 13:45:04.331072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:04.330710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" event={"ID":"b40b268c-02f6-422c-9b66-93c842f35095","Type":"ContainerDied","Data":"2998898461ec70511b12df82f4e1ee7a6ccf21039a4163b634592725e2fe474f"} Jul 14 13:45:05.468102 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:05.468077 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" Jul 14 13:45:06.342283 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:06.342247 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" event={"ID":"b40b268c-02f6-422c-9b66-93c842f35095","Type":"ContainerDied","Data":"c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023"} Jul 14 13:45:06.342283 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:06.342276 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733945-x82tt" Jul 14 13:45:06.342512 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:45:06.342298 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5afb61fead862c0139115f98ffeb2f581c540e7694140acc858ac5bd9b2c023" Jul 14 13:46:47.309954 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:46:47.309872 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:46:59.403266 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:46:59.403229 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:47:09.213387 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:09.213350 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:47:14.071488 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:14.071456 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:47:14.074086 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:14.074058 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:47:14.077619 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:14.077598 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:47:14.079771 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:14.079748 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:47:19.801662 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:19.801628 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:47:29.600447 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:29.600412 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:47:40.692291 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:40.692256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:47:51.219936 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:47:51.217301 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:48:00.699217 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:48:00.699178 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:48:12.683114 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:48:12.683023 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:48:22.098590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:48:22.098550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:48:33.999539 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:48:33.999504 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:48:59.188844 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:48:59.188808 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:49:06.398108 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:49:06.398073 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:49:16.495901 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:49:16.495868 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:49:25.895547 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:49:25.895514 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:49:36.108891 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:49:36.108794 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:49:50.014089 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:49:50.014053 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:00.092321 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:00.092279 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:09.700600 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:09.700564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:18.205370 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:18.205334 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:29.784563 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:29.784527 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:40.406095 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:40.406060 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:50:51.895073 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:50:51.895032 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:51:01.711444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:51:01.711405 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:51:16.981619 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:51:16.981541 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:51:26.792809 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:51:26.792765 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:51:39.685005 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:51:39.684971 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:51:46.297801 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:51:46.297766 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:52:01.295808 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:01.295772 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:52:13.988009 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:13.987977 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:52:14.102774 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:14.102741 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:52:14.106462 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:14.106429 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:52:14.110118 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:14.110092 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:52:14.115881 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:14.115854 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:52:27.887553 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:27.887514 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:52:46.167743 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.167704 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:52:46.168308 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.168283 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b40b268c-02f6-422c-9b66-93c842f35095" containerName="cleanup" Jul 14 13:52:46.168308 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.168302 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b40b268c-02f6-422c-9b66-93c842f35095" containerName="cleanup" Jul 14 13:52:46.168424 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.168404 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b40b268c-02f6-422c-9b66-93c842f35095" containerName="cleanup" Jul 14 13:52:46.171568 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.171548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:52:46.174429 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.174406 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:52:46.174543 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.174450 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:52:46.179621 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.179593 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:52:46.342551 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.342520 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4\" (UID: \"a4602f76-1abe-4a81-baad-f3b4879449a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:52:46.443281 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.443195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4\" (UID: \"a4602f76-1abe-4a81-baad-f3b4879449a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:52:46.443846 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.443827 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4\" (UID: \"a4602f76-1abe-4a81-baad-f3b4879449a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:52:46.484209 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.484179 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:52:46.617128 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:46.617104 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:52:46.618811 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:52:46.618786 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4602f76_1abe_4a81_baad_f3b4879449a4.slice/crio-8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b WatchSource:0}: Error finding container 8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b: Status 404 returned error can't find the container with id 8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b Jul 14 13:52:47.594338 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:47.594239 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerID="e9befd77a42922ab70d05630f954a03b138ec8858f65746b18f2e2280c203259" exitCode=7 Jul 14 13:52:47.594338 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:47.594300 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerDied","Data":"e9befd77a42922ab70d05630f954a03b138ec8858f65746b18f2e2280c203259"} Jul 14 13:52:47.594850 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:47.594378 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerStarted","Data":"8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b"} Jul 14 13:52:47.594850 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:47.594628 2568 scope.go:117] "RemoveContainer" containerID="e9befd77a42922ab70d05630f954a03b138ec8858f65746b18f2e2280c203259" Jul 14 13:52:48.601300 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:48.601268 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerID="0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621" exitCode=7 Jul 14 13:52:48.601708 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:48.601362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerDied","Data":"0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621"} Jul 14 13:52:48.601708 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:48.601427 2568 scope.go:117] "RemoveContainer" containerID="e9befd77a42922ab70d05630f954a03b138ec8858f65746b18f2e2280c203259" Jul 14 13:52:48.601708 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:48.601530 2568 scope.go:117] "RemoveContainer" containerID="0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621" Jul 14 13:52:48.601888 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:52:48.601787 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4_odh-ai-gateway-infra(a4602f76-1abe-4a81-baad-f3b4879449a4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" Jul 14 13:52:49.607664 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:49.607636 2568 scope.go:117] "RemoveContainer" containerID="0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621" Jul 14 13:52:49.608050 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:52:49.607845 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4_odh-ai-gateway-infra(a4602f76-1abe-4a81-baad-f3b4879449a4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" Jul 14 13:52:50.009722 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:52:50.009685 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:53:00.132515 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:00.132483 2568 scope.go:117] "RemoveContainer" containerID="0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621" Jul 14 13:53:00.664278 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:00.664247 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerStarted","Data":"2ba71f17cb92ec284c634195537f02856310576cf55f46ac0d6b735517225493"} Jul 14 13:53:00.691006 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:00.690913 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" podStartSLOduration=14.69089744 podStartE2EDuration="14.69089744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:53:00.689229048 +0000 UTC m=+1547.155356603" watchObservedRunningTime="2026-07-14 13:53:00.69089744 +0000 UTC m=+1547.157024993" Jul 14 13:53:01.671248 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:01.671216 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerID="2ba71f17cb92ec284c634195537f02856310576cf55f46ac0d6b735517225493" exitCode=0 Jul 14 13:53:01.671673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:01.671266 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerDied","Data":"2ba71f17cb92ec284c634195537f02856310576cf55f46ac0d6b735517225493"} Jul 14 13:53:01.671673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:01.671316 2568 scope.go:117] "RemoveContainer" containerID="0b522dfbe120b729899815e5e6b927b1415b73239f687fd17f2217ac9fe4e621" Jul 14 13:53:01.699072 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:01.699042 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:53:02.815750 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:02.815711 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:53:02.880929 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:02.880896 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca\") pod \"a4602f76-1abe-4a81-baad-f3b4879449a4\" (UID: \"a4602f76-1abe-4a81-baad-f3b4879449a4\") " Jul 14 13:53:02.881214 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:02.881191 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4602f76-1abe-4a81-baad-f3b4879449a4" (UID: "a4602f76-1abe-4a81-baad-f3b4879449a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:53:02.982567 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:02.982474 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4602f76-1abe-4a81-baad-f3b4879449a4-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:53:03.685188 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:03.685139 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" event={"ID":"a4602f76-1abe-4a81-baad-f3b4879449a4","Type":"ContainerDied","Data":"8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b"} Jul 14 13:53:03.685393 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:03.685250 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c6e962d16774c1e9b8c0a3faa29a62da84e4310e76418bb040e2aa92b22976b" Jul 14 13:53:03.685393 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:03.685370 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4" Jul 14 13:53:03.720238 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:03.720210 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:53:03.727406 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:03.727372 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-gsjk4"] Jul 14 13:53:04.138250 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:04.138220 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" path="/var/lib/kubelet/pods/a4602f76-1abe-4a81-baad-f3b4879449a4/volumes" Jul 14 13:53:36.222984 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.222952 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg"] Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223304 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223315 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223328 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223334 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223345 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223350 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223415 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.223444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.223422 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:53:36.226607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.226592 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:36.229682 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.229664 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:53:36.229782 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.229715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:53:36.239859 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.239836 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg"] Jul 14 13:53:36.370212 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.370180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg\" (UID: \"1e918f85-d93f-4047-ae4d-20586e7fa1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:36.470971 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.470920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg\" (UID: \"1e918f85-d93f-4047-ae4d-20586e7fa1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:36.471545 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.471527 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg\" (UID: \"1e918f85-d93f-4047-ae4d-20586e7fa1ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:36.537292 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.537264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:36.669824 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.669791 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg"] Jul 14 13:53:36.673494 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:53:36.673451 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e918f85_d93f_4047_ae4d_20586e7fa1ae.slice/crio-4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758 WatchSource:0}: Error finding container 4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758: Status 404 returned error can't find the container with id 4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758 Jul 14 13:53:36.859225 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.859195 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" event={"ID":"1e918f85-d93f-4047-ae4d-20586e7fa1ae","Type":"ContainerStarted","Data":"82d83510ef0934b22462a78e9c0b891aa79593ccb31dece071f547c5c8ae7359"} Jul 14 13:53:36.859367 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.859258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" event={"ID":"1e918f85-d93f-4047-ae4d-20586e7fa1ae","Type":"ContainerStarted","Data":"4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758"} Jul 14 13:53:36.878698 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:36.878656 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" podStartSLOduration=0.878640231 podStartE2EDuration="878.640231ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:53:36.877389284 +0000 UTC m=+1583.343516838" watchObservedRunningTime="2026-07-14 13:53:36.878640231 +0000 UTC m=+1583.344767786" Jul 14 13:53:37.865302 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:37.865265 2568 generic.go:356] "Generic (PLEG): container finished" podID="1e918f85-d93f-4047-ae4d-20586e7fa1ae" containerID="82d83510ef0934b22462a78e9c0b891aa79593ccb31dece071f547c5c8ae7359" exitCode=0 Jul 14 13:53:37.865752 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:37.865348 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" event={"ID":"1e918f85-d93f-4047-ae4d-20586e7fa1ae","Type":"ContainerDied","Data":"82d83510ef0934b22462a78e9c0b891aa79593ccb31dece071f547c5c8ae7359"} Jul 14 13:53:39.006281 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.006254 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:39.096135 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.096101 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca\") pod \"1e918f85-d93f-4047-ae4d-20586e7fa1ae\" (UID: \"1e918f85-d93f-4047-ae4d-20586e7fa1ae\") " Jul 14 13:53:39.096481 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.096454 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e918f85-d93f-4047-ae4d-20586e7fa1ae" (UID: "1e918f85-d93f-4047-ae4d-20586e7fa1ae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:53:39.197269 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.197184 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e918f85-d93f-4047-ae4d-20586e7fa1ae-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:53:39.876165 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.876130 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" Jul 14 13:53:39.876340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.876166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg" event={"ID":"1e918f85-d93f-4047-ae4d-20586e7fa1ae","Type":"ContainerDied","Data":"4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758"} Jul 14 13:53:39.876340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:39.876215 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4991d4458e3deaaec45b994bb1d62b9253bcde98413da4e899635cd7c8d64758" Jul 14 13:53:49.286808 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:53:49.286774 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:54:07.895963 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.895925 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:07.896372 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.896303 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e918f85-d93f-4047-ae4d-20586e7fa1ae" containerName="revoke-keys" Jul 14 13:54:07.896372 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.896316 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e918f85-d93f-4047-ae4d-20586e7fa1ae" containerName="revoke-keys" Jul 14 13:54:07.896444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.896382 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4602f76-1abe-4a81-baad-f3b4879449a4" containerName="revoke-keys" Jul 14 13:54:07.896444 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.896391 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e918f85-d93f-4047-ae4d-20586e7fa1ae" containerName="revoke-keys" Jul 14 13:54:07.899590 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.899574 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:07.902870 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.902853 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:54:07.902950 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.902934 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:54:07.909331 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.909306 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:07.945008 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:07.944975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx\" (UID: \"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:08.046018 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:08.045991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx\" (UID: \"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:08.046567 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:08.046549 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx\" (UID: \"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:08.211673 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:08.211593 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:08.360840 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:08.360775 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:08.363240 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:54:08.363213 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244bb7f0_2143_4ddd_862b_a2b1ca88dfbe.slice/crio-ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b WatchSource:0}: Error finding container ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b: Status 404 returned error can't find the container with id ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b Jul 14 13:54:09.015424 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:09.015375 2568 generic.go:356] "Generic (PLEG): container finished" podID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerID="43ed5743bf44ec3b4ae8c437ffcc10e018fca9d4f9fec1db725eaf89c757863b" exitCode=7 Jul 14 13:54:09.015839 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:09.015459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" event={"ID":"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe","Type":"ContainerDied","Data":"43ed5743bf44ec3b4ae8c437ffcc10e018fca9d4f9fec1db725eaf89c757863b"} Jul 14 13:54:09.015839 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:09.015512 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" event={"ID":"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe","Type":"ContainerStarted","Data":"ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b"} Jul 14 13:54:09.015839 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:09.015602 2568 scope.go:117] "RemoveContainer" containerID="43ed5743bf44ec3b4ae8c437ffcc10e018fca9d4f9fec1db725eaf89c757863b" Jul 14 13:54:10.022434 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:10.022337 2568 generic.go:356] "Generic (PLEG): container finished" podID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerID="4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d" exitCode=7 Jul 14 13:54:10.022434 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:10.022409 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" event={"ID":"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe","Type":"ContainerDied","Data":"4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d"} Jul 14 13:54:10.022955 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:10.022453 2568 scope.go:117] "RemoveContainer" containerID="43ed5743bf44ec3b4ae8c437ffcc10e018fca9d4f9fec1db725eaf89c757863b" Jul 14 13:54:10.022955 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:10.022698 2568 scope.go:117] "RemoveContainer" containerID="4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d" Jul 14 13:54:10.023052 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:54:10.022963 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx_odh-ai-gateway-infra(244bb7f0-2143-4ddd-862b-a2b1ca88dfbe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" Jul 14 13:54:11.028785 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:11.028759 2568 scope.go:117] "RemoveContainer" containerID="4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d" Jul 14 13:54:11.029147 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:54:11.028938 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx_odh-ai-gateway-infra(244bb7f0-2143-4ddd-862b-a2b1ca88dfbe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" Jul 14 13:54:11.188181 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:11.188144 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:54:22.131630 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:22.131596 2568 scope.go:117] "RemoveContainer" containerID="4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d" Jul 14 13:54:23.089369 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:23.089334 2568 generic.go:356] "Generic (PLEG): container finished" podID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerID="a5924ddfdb7793e34b031d3ef66be3ad70fc5b7552ab7afe8c43fb8a59520a31" exitCode=0 Jul 14 13:54:23.089527 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:23.089407 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" event={"ID":"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe","Type":"ContainerDied","Data":"a5924ddfdb7793e34b031d3ef66be3ad70fc5b7552ab7afe8c43fb8a59520a31"} Jul 14 13:54:23.089527 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:23.089467 2568 scope.go:117] "RemoveContainer" containerID="4209fd158f1dc3f03a448ed8321205f6a8b2469c74085f074ed23db3006bbd8d" Jul 14 13:54:24.126415 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:24.126374 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:24.230390 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:24.230368 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:24.288794 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:24.288755 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca\") pod \"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe\" (UID: \"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe\") " Jul 14 13:54:24.289107 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:24.289086 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" (UID: "244bb7f0-2143-4ddd-862b-a2b1ca88dfbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:54:24.389797 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:24.389688 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:54:25.103771 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:25.103723 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" Jul 14 13:54:25.104063 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:25.103763 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx" event={"ID":"244bb7f0-2143-4ddd-862b-a2b1ca88dfbe","Type":"ContainerDied","Data":"ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b"} Jul 14 13:54:25.104063 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:25.103824 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea4155347486843b43e1b5127126956e7e24cb5002f0f282e09a1676c04ee39b" Jul 14 13:54:25.140525 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:25.140482 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:25.146212 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:25.146190 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-n6tsx"] Jul 14 13:54:26.137946 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:26.137917 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" path="/var/lib/kubelet/pods/244bb7f0-2143-4ddd-862b-a2b1ca88dfbe/volumes" Jul 14 13:54:57.939931 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.939901 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm"] Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940261 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940271 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940297 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940303 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940360 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940366 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940369 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.940564 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.940376 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:54:57.943273 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.943258 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:54:57.946132 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.946111 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:54:57.946241 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.946218 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:54:57.951246 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.951223 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm"] Jul 14 13:54:57.974642 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:57.974608 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm\" (UID: \"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:54:58.075568 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:58.075531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm\" (UID: \"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:54:58.076215 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:58.076193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm\" (UID: \"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:54:58.253517 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:58.253434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:54:58.387339 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:58.387311 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm"] Jul 14 13:54:58.389306 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:54:58.389276 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37ab69b_b0a6_4c2b_8a7b_4d6c052d2f00.slice/crio-5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b WatchSource:0}: Error finding container 5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b: Status 404 returned error can't find the container with id 5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b Jul 14 13:54:59.269631 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:59.269596 2568 generic.go:356] "Generic (PLEG): container finished" podID="a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00" containerID="06b75559f8c4295c20e223f7e3ae8a38f81f5e76ee270c34e6d618f8862e95ef" exitCode=0 Jul 14 13:54:59.270041 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:59.269641 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" event={"ID":"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00","Type":"ContainerDied","Data":"06b75559f8c4295c20e223f7e3ae8a38f81f5e76ee270c34e6d618f8862e95ef"} Jul 14 13:54:59.270041 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:54:59.269691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" event={"ID":"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00","Type":"ContainerStarted","Data":"5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b"} Jul 14 13:55:00.402584 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:00.402561 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:55:00.496223 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:00.496192 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca\") pod \"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00\" (UID: \"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00\") " Jul 14 13:55:00.496534 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:00.496512 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00" (UID: "a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:55:00.597702 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:00.597670 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:55:01.280288 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:01.280260 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" Jul 14 13:55:01.280463 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:01.280292 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm" event={"ID":"a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00","Type":"ContainerDied","Data":"5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b"} Jul 14 13:55:01.280463 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:01.280334 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e11d4fb866a8a4237481db17b2323944b6f4c115c4060d5a61f9ef04236161b" Jul 14 13:55:10.986698 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:10.986659 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:55:20.938231 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938194 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:20.938707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938560 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:55:20.938707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938571 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="244bb7f0-2143-4ddd-862b-a2b1ca88dfbe" containerName="revoke-keys" Jul 14 13:55:20.938707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938599 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00" containerName="revoke-keys" Jul 14 13:55:20.938707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938606 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00" containerName="revoke-keys" Jul 14 13:55:20.938707 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.938668 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00" containerName="revoke-keys" Jul 14 13:55:20.941602 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.941586 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:20.944390 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.944369 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:55:20.944520 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.944400 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:55:20.951263 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:20.951238 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:21.075494 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:21.075453 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8\" (UID: \"df1d6a97-4d51-4bf3-b748-3b55b7148310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:21.176114 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:21.176065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8\" (UID: \"df1d6a97-4d51-4bf3-b748-3b55b7148310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:21.176776 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:21.176723 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8\" (UID: \"df1d6a97-4d51-4bf3-b748-3b55b7148310\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:21.252566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:21.252474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:21.387988 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:21.387948 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:21.389493 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:55:21.389470 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1d6a97_4d51_4bf3_b748_3b55b7148310.slice/crio-70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d WatchSource:0}: Error finding container 70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d: Status 404 returned error can't find the container with id 70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d Jul 14 13:55:22.395755 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:22.395699 2568 generic.go:356] "Generic (PLEG): container finished" podID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerID="dd8c0fb6d6da6401f35a61aa0c63d9fcb5ddb46011404aa5d73c977df5a1eb3f" exitCode=7 Jul 14 13:55:22.396156 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:22.395781 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerDied","Data":"dd8c0fb6d6da6401f35a61aa0c63d9fcb5ddb46011404aa5d73c977df5a1eb3f"} Jul 14 13:55:22.396156 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:22.395828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerStarted","Data":"70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d"} Jul 14 13:55:22.396156 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:22.395974 2568 scope.go:117] "RemoveContainer" containerID="dd8c0fb6d6da6401f35a61aa0c63d9fcb5ddb46011404aa5d73c977df5a1eb3f" Jul 14 13:55:23.403268 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:23.403237 2568 generic.go:356] "Generic (PLEG): container finished" podID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerID="76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4" exitCode=7 Jul 14 13:55:23.403752 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:23.403318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerDied","Data":"76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4"} Jul 14 13:55:23.403752 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:23.403388 2568 scope.go:117] "RemoveContainer" containerID="dd8c0fb6d6da6401f35a61aa0c63d9fcb5ddb46011404aa5d73c977df5a1eb3f" Jul 14 13:55:23.403752 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:23.403511 2568 scope.go:117] "RemoveContainer" containerID="76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4" Jul 14 13:55:23.403887 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:55:23.403751 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8_odh-ai-gateway-infra(df1d6a97-4d51-4bf3-b748-3b55b7148310)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" Jul 14 13:55:24.410435 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:24.410404 2568 scope.go:117] "RemoveContainer" containerID="76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4" Jul 14 13:55:24.410856 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:55:24.410576 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8_odh-ai-gateway-infra(df1d6a97-4d51-4bf3-b748-3b55b7148310)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" Jul 14 13:55:24.688347 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:24.688260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:55:39.131750 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:39.131661 2568 scope.go:117] "RemoveContainer" containerID="76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4" Jul 14 13:55:39.487480 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:39.487447 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerStarted","Data":"21c9e2e0aaf206553fbd6ce54a016fad679a17556354c692cac7f58ab621a167"} Jul 14 13:55:39.507445 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:39.507395 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" podStartSLOduration=19.507378224 podStartE2EDuration="19.507378224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:39.505145398 +0000 UTC m=+1705.971272965" watchObservedRunningTime="2026-07-14 13:55:39.507378224 +0000 UTC m=+1705.973505778" Jul 14 13:55:40.494414 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:40.494383 2568 generic.go:356] "Generic (PLEG): container finished" podID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerID="21c9e2e0aaf206553fbd6ce54a016fad679a17556354c692cac7f58ab621a167" exitCode=0 Jul 14 13:55:40.494841 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:40.494457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerDied","Data":"21c9e2e0aaf206553fbd6ce54a016fad679a17556354c692cac7f58ab621a167"} Jul 14 13:55:40.494841 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:40.494518 2568 scope.go:117] "RemoveContainer" containerID="76b445146affbd9a6a009a1b49db793b1098cabde8698ccae472f21374cf12b4" Jul 14 13:55:40.525129 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:40.525101 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:41.637337 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:41.637307 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:41.661837 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:41.661808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca\") pod \"df1d6a97-4d51-4bf3-b748-3b55b7148310\" (UID: \"df1d6a97-4d51-4bf3-b748-3b55b7148310\") " Jul 14 13:55:41.662252 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:41.662221 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df1d6a97-4d51-4bf3-b748-3b55b7148310" (UID: "df1d6a97-4d51-4bf3-b748-3b55b7148310"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:55:41.762570 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:41.762483 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df1d6a97-4d51-4bf3-b748-3b55b7148310-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:55:42.507796 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:42.507708 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" Jul 14 13:55:42.507796 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:42.507756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8" event={"ID":"df1d6a97-4d51-4bf3-b748-3b55b7148310","Type":"ContainerDied","Data":"70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d"} Jul 14 13:55:42.507990 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:42.507802 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70501c0106e72aaa05e90e366fd456083c94d313df6dafe32afbc9b03c4fd99d" Jul 14 13:55:42.540764 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:42.540709 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:42.547382 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:42.547357 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-cvkn8"] Jul 14 13:55:44.137682 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:55:44.137653 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" path="/var/lib/kubelet/pods/df1d6a97-4d51-4bf3-b748-3b55b7148310/volumes" Jul 14 13:56:20.998218 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998184 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq"] Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998556 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998567 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998576 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998582 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998590 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998595 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998660 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:20.998692 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:20.998667 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:21.001541 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.001525 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:21.004340 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.004317 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:56:21.004431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.004317 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:56:21.013365 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.013342 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq"] Jul 14 13:56:21.122675 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.122640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq\" (UID: \"55e4606c-7066-4f9f-8419-dce730c990ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:21.223566 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.223518 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq\" (UID: \"55e4606c-7066-4f9f-8419-dce730c990ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:21.224240 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.224213 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq\" (UID: \"55e4606c-7066-4f9f-8419-dce730c990ee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:21.312524 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.312480 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:21.448082 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.448055 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq"] Jul 14 13:56:21.450274 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:56:21.450251 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e4606c_7066_4f9f_8419_dce730c990ee.slice/crio-60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2 WatchSource:0}: Error finding container 60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2: Status 404 returned error can't find the container with id 60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2 Jul 14 13:56:21.705142 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:21.705051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" event={"ID":"55e4606c-7066-4f9f-8419-dce730c990ee","Type":"ContainerStarted","Data":"60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2"} Jul 14 13:56:22.713524 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:22.713492 2568 generic.go:356] "Generic (PLEG): container finished" podID="55e4606c-7066-4f9f-8419-dce730c990ee" containerID="ab6c09855d8a3d286ad92b40d4f5cfbbfaa4322aebea397870317bf3ed631dc5" exitCode=0 Jul 14 13:56:22.713987 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:22.713541 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" event={"ID":"55e4606c-7066-4f9f-8419-dce730c990ee","Type":"ContainerDied","Data":"ab6c09855d8a3d286ad92b40d4f5cfbbfaa4322aebea397870317bf3ed631dc5"} Jul 14 13:56:23.849608 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:23.849580 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:23.943557 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:23.943518 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca\") pod \"55e4606c-7066-4f9f-8419-dce730c990ee\" (UID: \"55e4606c-7066-4f9f-8419-dce730c990ee\") " Jul 14 13:56:23.943903 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:23.943880 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55e4606c-7066-4f9f-8419-dce730c990ee" (UID: "55e4606c-7066-4f9f-8419-dce730c990ee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:24.044703 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:24.044672 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55e4606c-7066-4f9f-8419-dce730c990ee-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:56:24.724405 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:24.724381 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" Jul 14 13:56:24.724581 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:24.724429 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq" event={"ID":"55e4606c-7066-4f9f-8419-dce730c990ee","Type":"ContainerDied","Data":"60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2"} Jul 14 13:56:24.724581 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:24.724473 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60f4035c2a8f24ebc9477c3fe2cef84d6ee92ba4bc4d2d4e64bb9efe1dbeb1a2" Jul 14 13:56:34.488135 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:34.488101 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:56:51.588784 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:51.588749 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:56:53.090258 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.090222 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:56:53.090658 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.090600 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e4606c-7066-4f9f-8419-dce730c990ee" containerName="revoke-keys" Jul 14 13:56:53.090658 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.090611 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e4606c-7066-4f9f-8419-dce730c990ee" containerName="revoke-keys" Jul 14 13:56:53.090751 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.090685 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e4606c-7066-4f9f-8419-dce730c990ee" containerName="revoke-keys" Jul 14 13:56:53.090751 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.090695 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="df1d6a97-4d51-4bf3-b748-3b55b7148310" containerName="revoke-keys" Jul 14 13:56:53.093782 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.093766 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:56:53.096607 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.096585 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:56:53.096748 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.096674 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:56:53.102462 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.102441 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:56:53.179762 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.179694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d\" (UID: \"c37d4a0a-651f-4326-819b-652345ca46ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:56:53.280659 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.280619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d\" (UID: \"c37d4a0a-651f-4326-819b-652345ca46ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:56:53.281236 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.281214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d\" (UID: \"c37d4a0a-651f-4326-819b-652345ca46ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:56:53.404928 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.404840 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:56:53.545605 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.545582 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:56:53.547513 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:56:53.547486 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37d4a0a_651f_4326_819b_652345ca46ed.slice/crio-701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745 WatchSource:0}: Error finding container 701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745: Status 404 returned error can't find the container with id 701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745 Jul 14 13:56:53.869284 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.869249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerStarted","Data":"8f5779265dfdd917b0d3531d0e541fbe535bffabfe4b39c2bc7ec74365125a23"} Jul 14 13:56:53.869431 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.869307 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerStarted","Data":"701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745"} Jul 14 13:56:53.888829 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:53.888784 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" podStartSLOduration=0.888769676 podStartE2EDuration="888.769676ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:56:53.88590528 +0000 UTC m=+1780.352032834" watchObservedRunningTime="2026-07-14 13:56:53.888769676 +0000 UTC m=+1780.354897208" Jul 14 13:56:54.876521 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:54.876482 2568 generic.go:356] "Generic (PLEG): container finished" podID="c37d4a0a-651f-4326-819b-652345ca46ed" containerID="8f5779265dfdd917b0d3531d0e541fbe535bffabfe4b39c2bc7ec74365125a23" exitCode=7 Jul 14 13:56:54.876989 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:54.876573 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerDied","Data":"8f5779265dfdd917b0d3531d0e541fbe535bffabfe4b39c2bc7ec74365125a23"} Jul 14 13:56:54.876989 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:54.876781 2568 scope.go:117] "RemoveContainer" containerID="8f5779265dfdd917b0d3531d0e541fbe535bffabfe4b39c2bc7ec74365125a23" Jul 14 13:56:55.883628 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:55.883593 2568 generic.go:356] "Generic (PLEG): container finished" podID="c37d4a0a-651f-4326-819b-652345ca46ed" containerID="d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788" exitCode=7 Jul 14 13:56:55.884039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:55.883697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerDied","Data":"d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788"} Jul 14 13:56:55.884039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:55.883775 2568 scope.go:117] "RemoveContainer" containerID="8f5779265dfdd917b0d3531d0e541fbe535bffabfe4b39c2bc7ec74365125a23" Jul 14 13:56:55.884039 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:55.883993 2568 scope.go:117] "RemoveContainer" containerID="d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788" Jul 14 13:56:55.884232 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:56:55.884220 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d_odh-ai-gateway-infra(c37d4a0a-651f-4326-819b-652345ca46ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" Jul 14 13:56:56.893557 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:56:56.893529 2568 scope.go:117] "RemoveContainer" containerID="d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788" Jul 14 13:56:56.893970 ip-10-0-138-114 kubenswrapper[2568]: E0714 13:56:56.893753 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d_odh-ai-gateway-infra(c37d4a0a-651f-4326-819b-652345ca46ed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" Jul 14 13:57:12.132426 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:12.132389 2568 scope.go:117] "RemoveContainer" containerID="d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788" Jul 14 13:57:12.983924 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:12.983891 2568 generic.go:356] "Generic (PLEG): container finished" podID="c37d4a0a-651f-4326-819b-652345ca46ed" containerID="95d573809182513c0f14fd92e4b511bd1d5b6992f0712dabcc12a95294bd033c" exitCode=0 Jul 14 13:57:12.984079 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:12.983970 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerDied","Data":"95d573809182513c0f14fd92e4b511bd1d5b6992f0712dabcc12a95294bd033c"} Jul 14 13:57:12.984079 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:12.984031 2568 scope.go:117] "RemoveContainer" containerID="d772e464c7ae88e51f6dc6cb5a1397a966e52e41b835573ccc8c33eddfed4788" Jul 14 13:57:14.015098 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.015063 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:57:14.139100 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.139023 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:57:14.145862 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.145829 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:57:14.148985 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.148960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 13:57:14.151424 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.151405 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:57:14.154764 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.154747 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 13:57:14.271239 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.271123 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca\") pod \"c37d4a0a-651f-4326-819b-652345ca46ed\" (UID: \"c37d4a0a-651f-4326-819b-652345ca46ed\") " Jul 14 13:57:14.271550 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.271525 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c37d4a0a-651f-4326-819b-652345ca46ed" (UID: "c37d4a0a-651f-4326-819b-652345ca46ed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:14.372536 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.372484 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c37d4a0a-651f-4326-819b-652345ca46ed-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:57:14.996697 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.996667 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" Jul 14 13:57:14.996926 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.996666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d" event={"ID":"c37d4a0a-651f-4326-819b-652345ca46ed","Type":"ContainerDied","Data":"701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745"} Jul 14 13:57:14.996926 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:14.996800 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="701070e796d6f1f85feeaa78f84acb2bc0bee61226652fd5a139302e46dd9745" Jul 14 13:57:15.032817 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:15.032772 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:57:15.039183 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:15.039155 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-9lg9d"] Jul 14 13:57:16.137187 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:16.137155 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" path="/var/lib/kubelet/pods/c37d4a0a-651f-4326-819b-652345ca46ed/volumes" Jul 14 13:57:53.140630 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.140591 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx"] Jul 14 13:57:53.141161 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141126 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141161 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141145 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141282 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141179 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141282 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141190 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141282 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141207 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141282 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141216 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141480 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141306 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141480 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141316 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.141480 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.141327 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37d4a0a-651f-4326-819b-652345ca46ed" containerName="revoke-keys" Jul 14 13:57:53.144262 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.144241 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:53.147256 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.147238 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 13:57:53.147367 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.147253 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 13:57:53.152702 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.152681 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx"] Jul 14 13:57:53.211487 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.211444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx\" (UID: \"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:53.312912 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.312878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx\" (UID: \"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:53.313558 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.313533 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx\" (UID: \"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:53.456592 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.456485 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:53.590352 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:53.590324 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx"] Jul 14 13:57:53.592235 ip-10-0-138-114 kubenswrapper[2568]: W0714 13:57:53.592210 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae82ba5_e5e9_46bf_97b2_f7e80bf8f6e8.slice/crio-37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d WatchSource:0}: Error finding container 37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d: Status 404 returned error can't find the container with id 37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d Jul 14 13:57:54.191369 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:54.191278 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" event={"ID":"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8","Type":"ContainerStarted","Data":"74e26a2e79f0bb36de6c0a2c2c3261799f13f4c8928328ba3801ad6565515d82"} Jul 14 13:57:54.191369 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:54.191339 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" event={"ID":"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8","Type":"ContainerStarted","Data":"37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d"} Jul 14 13:57:54.213544 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:54.213491 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" podStartSLOduration=1.213473604 podStartE2EDuration="1.213473604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:57:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:57:54.209849254 +0000 UTC m=+1840.675976807" watchObservedRunningTime="2026-07-14 13:57:54.213473604 +0000 UTC m=+1840.679601136" Jul 14 13:57:55.198236 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:55.198199 2568 generic.go:356] "Generic (PLEG): container finished" podID="7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8" containerID="74e26a2e79f0bb36de6c0a2c2c3261799f13f4c8928328ba3801ad6565515d82" exitCode=0 Jul 14 13:57:55.198618 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:55.198286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" event={"ID":"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8","Type":"ContainerDied","Data":"74e26a2e79f0bb36de6c0a2c2c3261799f13f4c8928328ba3801ad6565515d82"} Jul 14 13:57:56.340592 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:56.340566 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:56.448167 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:56.448129 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca\") pod \"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8\" (UID: \"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8\") " Jul 14 13:57:56.448588 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:56.448562 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8" (UID: "7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:56.548921 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:56.548884 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 13:57:57.210962 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:57.210928 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" Jul 14 13:57:57.211107 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:57.210963 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx" event={"ID":"7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8","Type":"ContainerDied","Data":"37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d"} Jul 14 13:57:57.211107 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:57:57.211012 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37369baedb478a24076fcd8baf71690551ba57f0733fef25f6677968a2100b8d" Jul 14 13:58:06.586656 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:58:06.586621 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:59:14.195086 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:59:14.195054 2568 scope.go:117] "RemoveContainer" containerID="2ba71f17cb92ec284c634195537f02856310576cf55f46ac0d6b735517225493" Jul 14 13:59:43.679623 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:59:43.679584 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 13:59:52.883652 ip-10-0-138-114 kubenswrapper[2568]: I0714 13:59:52.883619 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:00:00.144304 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.144269 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv"] Jul 14 14:00:00.144805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.144630 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8" containerName="revoke-keys" Jul 14 14:00:00.144805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.144642 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8" containerName="revoke-keys" Jul 14 14:00:00.144805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.144740 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8" containerName="revoke-keys" Jul 14 14:00:00.148193 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.148170 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv"] Jul 14 14:00:00.148316 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.148252 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" Jul 14 14:00:00.151064 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.151038 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:00:00.157761 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.157744 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" Jul 14 14:00:00.292421 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.292394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv"] Jul 14 14:00:00.294585 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:00:00.294561 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ca454f8_d8d6_4378_852a_4dbb7f884106.slice/crio-e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b WatchSource:0}: Error finding container e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b: Status 404 returned error can't find the container with id e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b Jul 14 14:00:00.861237 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.861148 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" event={"ID":"0ca454f8-d8d6-4378-852a-4dbb7f884106","Type":"ContainerStarted","Data":"735214fde805469a5f84e2a9d6844dceec8df47701471940c5da60d3baa02fc3"} Jul 14 14:00:00.861237 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.861213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" event={"ID":"0ca454f8-d8d6-4378-852a-4dbb7f884106","Type":"ContainerStarted","Data":"e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b"} Jul 14 14:00:00.880334 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:00.880281 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" podStartSLOduration=0.880264489 podStartE2EDuration="880.264489ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:00:00.87843424 +0000 UTC m=+1967.344561820" watchObservedRunningTime="2026-07-14 14:00:00.880264489 +0000 UTC m=+1967.346392043" Jul 14 14:00:01.868633 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:01.868593 2568 generic.go:356] "Generic (PLEG): container finished" podID="0ca454f8-d8d6-4378-852a-4dbb7f884106" containerID="735214fde805469a5f84e2a9d6844dceec8df47701471940c5da60d3baa02fc3" exitCode=0 Jul 14 14:00:01.869157 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:01.868684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" event={"ID":"0ca454f8-d8d6-4378-852a-4dbb7f884106","Type":"ContainerDied","Data":"735214fde805469a5f84e2a9d6844dceec8df47701471940c5da60d3baa02fc3"} Jul 14 14:00:03.011424 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:03.011402 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" Jul 14 14:00:03.880173 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:03.880140 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" event={"ID":"0ca454f8-d8d6-4378-852a-4dbb7f884106","Type":"ContainerDied","Data":"e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b"} Jul 14 14:00:03.880173 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:03.880161 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733960-c52gv" Jul 14 14:00:03.880397 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:03.880187 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e759ee05297cb44a7f6b22c40a1939ae921f166b4e61c201cac606db65f2336b" Jul 14 14:00:05.295609 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:05.295524 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:00:12.300142 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.300106 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh"] Jul 14 14:00:12.300523 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.300509 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ca454f8-d8d6-4378-852a-4dbb7f884106" containerName="cleanup" Jul 14 14:00:12.300523 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.300524 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ca454f8-d8d6-4378-852a-4dbb7f884106" containerName="cleanup" Jul 14 14:00:12.300605 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.300597 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ca454f8-d8d6-4378-852a-4dbb7f884106" containerName="cleanup" Jul 14 14:00:12.303807 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.303790 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:12.306402 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.306385 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:00:12.306474 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.306426 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:00:12.311327 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.311231 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh"] Jul 14 14:00:12.434564 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.434526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-50aeb779-w79mh\" (UID: \"84a814ef-89aa-4545-85a4-b9a77520f4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:12.536095 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.536056 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-50aeb779-w79mh\" (UID: \"84a814ef-89aa-4545-85a4-b9a77520f4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:12.536709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.536689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-50aeb779-w79mh\" (UID: \"84a814ef-89aa-4545-85a4-b9a77520f4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:12.615218 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.615135 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:12.755401 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.755374 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh"] Jul 14 14:00:12.757694 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:00:12.757669 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a814ef_89aa_4545_85a4_b9a77520f4f3.slice/crio-7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e WatchSource:0}: Error finding container 7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e: Status 404 returned error can't find the container with id 7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e Jul 14 14:00:12.930903 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:12.930876 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" event={"ID":"84a814ef-89aa-4545-85a4-b9a77520f4f3","Type":"ContainerStarted","Data":"7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e"} Jul 14 14:00:13.937672 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:13.937637 2568 generic.go:356] "Generic (PLEG): container finished" podID="84a814ef-89aa-4545-85a4-b9a77520f4f3" containerID="734f5d38b9a53ef6d0bf5281105e07fab12102499285183a2201e8d122257d4a" exitCode=0 Jul 14 14:00:13.938088 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:13.937718 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" event={"ID":"84a814ef-89aa-4545-85a4-b9a77520f4f3","Type":"ContainerDied","Data":"734f5d38b9a53ef6d0bf5281105e07fab12102499285183a2201e8d122257d4a"} Jul 14 14:00:15.081513 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.081484 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:15.158672 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.158637 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca\") pod \"84a814ef-89aa-4545-85a4-b9a77520f4f3\" (UID: \"84a814ef-89aa-4545-85a4-b9a77520f4f3\") " Jul 14 14:00:15.159021 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.158996 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84a814ef-89aa-4545-85a4-b9a77520f4f3" (UID: "84a814ef-89aa-4545-85a4-b9a77520f4f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:00:15.259695 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.259600 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84a814ef-89aa-4545-85a4-b9a77520f4f3-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:00:15.949720 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.949692 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" Jul 14 14:00:15.949918 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.949697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-50aeb779-w79mh" event={"ID":"84a814ef-89aa-4545-85a4-b9a77520f4f3","Type":"ContainerDied","Data":"7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e"} Jul 14 14:00:15.949918 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:15.949816 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7808ea6e98b4082034d65f3e59691da2c14fdd7a7690fad3811dd2097bd3b25e" Jul 14 14:00:25.676354 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:00:25.676321 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:01:07.391375 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:07.391339 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:01:14.222638 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:14.222601 2568 scope.go:117] "RemoveContainer" containerID="a5924ddfdb7793e34b031d3ef66be3ad70fc5b7552ab7afe8c43fb8a59520a31" Jul 14 14:01:23.684298 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:23.684266 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:01:38.888941 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:38.888853 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:01:45.009894 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.009858 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk"] Jul 14 14:01:45.010288 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.010241 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a814ef-89aa-4545-85a4-b9a77520f4f3" containerName="revoke-keys" Jul 14 14:01:45.010288 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.010252 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a814ef-89aa-4545-85a4-b9a77520f4f3" containerName="revoke-keys" Jul 14 14:01:45.010375 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.010316 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a814ef-89aa-4545-85a4-b9a77520f4f3" containerName="revoke-keys" Jul 14 14:01:45.013595 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.013579 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:45.016512 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.016490 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:01:45.016647 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.016493 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:01:45.022575 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.022552 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk"] Jul 14 14:01:45.070179 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.070124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk\" (UID: \"32c8756d-24f3-499f-b01f-25f9ee281d67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:45.171371 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.171338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk\" (UID: \"32c8756d-24f3-499f-b01f-25f9ee281d67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:45.171965 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.171939 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk\" (UID: \"32c8756d-24f3-499f-b01f-25f9ee281d67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:45.324827 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.324796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:45.460202 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:45.460178 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk"] Jul 14 14:01:45.462316 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:01:45.462290 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c8756d_24f3_499f_b01f_25f9ee281d67.slice/crio-51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb WatchSource:0}: Error finding container 51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb: Status 404 returned error can't find the container with id 51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb Jul 14 14:01:46.427240 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:46.427146 2568 generic.go:356] "Generic (PLEG): container finished" podID="32c8756d-24f3-499f-b01f-25f9ee281d67" containerID="95850621fc6a0ffb971102dbf9eb0ec6f25e2ce027d2e46687b5626a8a4d61fe" exitCode=0 Jul 14 14:01:46.427618 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:46.427237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" event={"ID":"32c8756d-24f3-499f-b01f-25f9ee281d67","Type":"ContainerDied","Data":"95850621fc6a0ffb971102dbf9eb0ec6f25e2ce027d2e46687b5626a8a4d61fe"} Jul 14 14:01:46.427618 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:46.427302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" event={"ID":"32c8756d-24f3-499f-b01f-25f9ee281d67","Type":"ContainerStarted","Data":"51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb"} Jul 14 14:01:47.563402 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:47.563375 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:47.693176 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:47.693083 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca\") pod \"32c8756d-24f3-499f-b01f-25f9ee281d67\" (UID: \"32c8756d-24f3-499f-b01f-25f9ee281d67\") " Jul 14 14:01:47.693458 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:47.693434 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "32c8756d-24f3-499f-b01f-25f9ee281d67" (UID: "32c8756d-24f3-499f-b01f-25f9ee281d67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:01:47.794075 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:47.794043 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32c8756d-24f3-499f-b01f-25f9ee281d67-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:01:48.439651 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:48.439565 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" Jul 14 14:01:48.439815 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:48.439570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk" event={"ID":"32c8756d-24f3-499f-b01f-25f9ee281d67","Type":"ContainerDied","Data":"51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb"} Jul 14 14:01:48.439815 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:48.439696 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51f380e0cbc3a96c54b42bfd1b3c38be6c8fa0e1009d06eea3d92176e21e25cb" Jul 14 14:01:59.085599 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:01:59.085557 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:02:14.179399 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:02:14.179363 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:02:14.185165 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:02:14.185139 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:02:14.185329 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:02:14.185196 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:02:14.190919 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:02:14.190897 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:02:14.242188 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:02:14.242157 2568 scope.go:117] "RemoveContainer" containerID="21c9e2e0aaf206553fbd6ce54a016fad679a17556354c692cac7f58ab621a167" Jul 14 14:03:06.583927 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:03:06.583847 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:03:14.265961 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:03:14.265921 2568 scope.go:117] "RemoveContainer" containerID="95d573809182513c0f14fd92e4b511bd1d5b6992f0712dabcc12a95294bd033c" Jul 14 14:03:21.121668 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:03:21.121626 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:04:10.881034 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:04:10.880994 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:04:25.482205 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:04:25.482165 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:04:40.885053 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:04:40.884968 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:04:52.786115 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:04:52.786080 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:05:34.080923 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:05:34.080888 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:05:49.083517 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:05:49.083481 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:06:06.362061 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.361980 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj"] Jul 14 14:06:06.362509 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.362365 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32c8756d-24f3-499f-b01f-25f9ee281d67" containerName="revoke-keys" Jul 14 14:06:06.362509 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.362376 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="32c8756d-24f3-499f-b01f-25f9ee281d67" containerName="revoke-keys" Jul 14 14:06:06.362509 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.362442 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="32c8756d-24f3-499f-b01f-25f9ee281d67" containerName="revoke-keys" Jul 14 14:06:06.365640 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.365622 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:06.368371 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.368348 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:06:06.368493 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.368448 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:06:06.374526 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.374497 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj"] Jul 14 14:06:06.419205 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.419174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj\" (UID: \"33f19320-1639-48e1-b9dd-e8ce407dc5bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:06.519921 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.519884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj\" (UID: \"33f19320-1639-48e1-b9dd-e8ce407dc5bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:06.520492 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.520471 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj\" (UID: \"33f19320-1639-48e1-b9dd-e8ce407dc5bd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:06.677216 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.677123 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:06.817034 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:06.817010 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj"] Jul 14 14:06:06.823228 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:06:06.823186 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f19320_1639_48e1_b9dd_e8ce407dc5bd.slice/crio-6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3 WatchSource:0}: Error finding container 6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3: Status 404 returned error can't find the container with id 6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3 Jul 14 14:06:07.813172 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:07.813142 2568 generic.go:356] "Generic (PLEG): container finished" podID="33f19320-1639-48e1-b9dd-e8ce407dc5bd" containerID="07a31821a25d4851c1b000d1cd4c2ef849abdbddbc7b283e755c3cc9e8905025" exitCode=0 Jul 14 14:06:07.813572 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:07.813230 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" event={"ID":"33f19320-1639-48e1-b9dd-e8ce407dc5bd","Type":"ContainerDied","Data":"07a31821a25d4851c1b000d1cd4c2ef849abdbddbc7b283e755c3cc9e8905025"} Jul 14 14:06:07.813572 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:07.813287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" event={"ID":"33f19320-1639-48e1-b9dd-e8ce407dc5bd","Type":"ContainerStarted","Data":"6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3"} Jul 14 14:06:08.952549 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:08.952525 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:09.043072 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.043033 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca\") pod \"33f19320-1639-48e1-b9dd-e8ce407dc5bd\" (UID: \"33f19320-1639-48e1-b9dd-e8ce407dc5bd\") " Jul 14 14:06:09.043432 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.043407 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33f19320-1639-48e1-b9dd-e8ce407dc5bd" (UID: "33f19320-1639-48e1-b9dd-e8ce407dc5bd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:06:09.144083 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.143996 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33f19320-1639-48e1-b9dd-e8ce407dc5bd-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:06:09.824399 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.824370 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" Jul 14 14:06:09.824576 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.824366 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj" event={"ID":"33f19320-1639-48e1-b9dd-e8ce407dc5bd","Type":"ContainerDied","Data":"6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3"} Jul 14 14:06:09.824576 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:09.824493 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ddcfaed0e98bc2a0cc95516c7a3f0bea17217181d96441541c9ccafd4e345b3" Jul 14 14:06:10.381256 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:10.381220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:06:27.473597 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:27.473557 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:06:32.485849 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.485812 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd"] Jul 14 14:06:32.486233 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.486224 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33f19320-1639-48e1-b9dd-e8ce407dc5bd" containerName="revoke-keys" Jul 14 14:06:32.486279 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.486235 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="33f19320-1639-48e1-b9dd-e8ce407dc5bd" containerName="revoke-keys" Jul 14 14:06:32.486317 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.486298 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="33f19320-1639-48e1-b9dd-e8ce407dc5bd" containerName="revoke-keys" Jul 14 14:06:32.489454 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.489434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:32.492391 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.492366 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:06:32.492500 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.492376 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:06:32.503429 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.503404 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd"] Jul 14 14:06:32.570864 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.570822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd\" (UID: \"39174521-7e2e-4604-a56d-4a2e7789e1c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:32.672352 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.672311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd\" (UID: \"39174521-7e2e-4604-a56d-4a2e7789e1c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:32.673019 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.672999 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd\" (UID: \"39174521-7e2e-4604-a56d-4a2e7789e1c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:32.799251 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.799218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:32.941030 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:32.940966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd"] Jul 14 14:06:32.943696 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:06:32.943662 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39174521_7e2e_4604_a56d_4a2e7789e1c3.slice/crio-8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da WatchSource:0}: Error finding container 8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da: Status 404 returned error can't find the container with id 8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da Jul 14 14:06:33.950221 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:33.950127 2568 generic.go:356] "Generic (PLEG): container finished" podID="39174521-7e2e-4604-a56d-4a2e7789e1c3" containerID="3062786788a32901a9a81515da13ca37afa70bd61d5fbd7422c07500bb0517f5" exitCode=0 Jul 14 14:06:33.950632 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:33.950218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" event={"ID":"39174521-7e2e-4604-a56d-4a2e7789e1c3","Type":"ContainerDied","Data":"3062786788a32901a9a81515da13ca37afa70bd61d5fbd7422c07500bb0517f5"} Jul 14 14:06:33.950632 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:33.950268 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" event={"ID":"39174521-7e2e-4604-a56d-4a2e7789e1c3","Type":"ContainerStarted","Data":"8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da"} Jul 14 14:06:35.091255 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.091230 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:35.196796 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.196761 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca\") pod \"39174521-7e2e-4604-a56d-4a2e7789e1c3\" (UID: \"39174521-7e2e-4604-a56d-4a2e7789e1c3\") " Jul 14 14:06:35.197132 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.197108 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39174521-7e2e-4604-a56d-4a2e7789e1c3" (UID: "39174521-7e2e-4604-a56d-4a2e7789e1c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:06:35.298336 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.298299 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39174521-7e2e-4604-a56d-4a2e7789e1c3-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:06:35.961773 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.961720 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" Jul 14 14:06:35.961921 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.961770 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd" event={"ID":"39174521-7e2e-4604-a56d-4a2e7789e1c3","Type":"ContainerDied","Data":"8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da"} Jul 14 14:06:35.961921 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:35.961820 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f9997b7328e145caed73d622a4f01a0ba75c6663ee2047847893b5bda3861da" Jul 14 14:06:46.183704 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:06:46.183666 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:07:09.387019 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:09.386978 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:07:14.215300 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:14.215259 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:07:14.221185 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:14.221154 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:07:14.223036 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:14.223013 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:07:14.228426 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:14.228407 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:07:22.710423 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.710381 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65"] Jul 14 14:07:22.710929 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.710816 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39174521-7e2e-4604-a56d-4a2e7789e1c3" containerName="revoke-keys" Jul 14 14:07:22.710929 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.710831 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="39174521-7e2e-4604-a56d-4a2e7789e1c3" containerName="revoke-keys" Jul 14 14:07:22.710929 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.710892 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="39174521-7e2e-4604-a56d-4a2e7789e1c3" containerName="revoke-keys" Jul 14 14:07:22.713968 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.713946 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:22.717379 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.717358 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:07:22.717521 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.717362 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:07:22.722642 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.722612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65"] Jul 14 14:07:22.826009 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.825970 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f2a083-vqq65\" (UID: \"febca3b0-b3ef-419a-aaf6-c0a0da0df740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:22.926522 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.926484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f2a083-vqq65\" (UID: \"febca3b0-b3ef-419a-aaf6-c0a0da0df740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:22.927185 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:22.927163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f2a083-vqq65\" (UID: \"febca3b0-b3ef-419a-aaf6-c0a0da0df740\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:23.025834 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:23.025751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:23.167337 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:23.167307 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65"] Jul 14 14:07:23.169429 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:07:23.169400 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfebca3b0_b3ef_419a_aaf6_c0a0da0df740.slice/crio-0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be WatchSource:0}: Error finding container 0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be: Status 404 returned error can't find the container with id 0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be Jul 14 14:07:23.213700 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:23.213659 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" event={"ID":"febca3b0-b3ef-419a-aaf6-c0a0da0df740","Type":"ContainerStarted","Data":"0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be"} Jul 14 14:07:24.220451 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:24.220351 2568 generic.go:356] "Generic (PLEG): container finished" podID="febca3b0-b3ef-419a-aaf6-c0a0da0df740" containerID="10bf57a5f5842265a959129260f20b68fc2a0a374536f51b4284c8fd6d94cb4b" exitCode=0 Jul 14 14:07:24.220451 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:24.220400 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" event={"ID":"febca3b0-b3ef-419a-aaf6-c0a0da0df740","Type":"ContainerDied","Data":"10bf57a5f5842265a959129260f20b68fc2a0a374536f51b4284c8fd6d94cb4b"} Jul 14 14:07:25.289537 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:25.289498 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:07:25.362299 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:25.362273 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:25.450637 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:25.450606 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca\") pod \"febca3b0-b3ef-419a-aaf6-c0a0da0df740\" (UID: \"febca3b0-b3ef-419a-aaf6-c0a0da0df740\") " Jul 14 14:07:25.451088 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:25.451069 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "febca3b0-b3ef-419a-aaf6-c0a0da0df740" (UID: "febca3b0-b3ef-419a-aaf6-c0a0da0df740"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:07:25.551918 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:25.551890 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/febca3b0-b3ef-419a-aaf6-c0a0da0df740-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:07:26.232373 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:26.232343 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" Jul 14 14:07:26.232519 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:26.232382 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f2a083-vqq65" event={"ID":"febca3b0-b3ef-419a-aaf6-c0a0da0df740","Type":"ContainerDied","Data":"0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be"} Jul 14 14:07:26.232519 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:26.232433 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b581756033464705192caaa072d75b6d41e79bdc5b2f0a0e76813946bf7f7be" Jul 14 14:07:36.883533 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:36.883442 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:07:48.844941 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.844903 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq"] Jul 14 14:07:48.845314 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.845276 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="febca3b0-b3ef-419a-aaf6-c0a0da0df740" containerName="revoke-keys" Jul 14 14:07:48.845314 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.845287 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="febca3b0-b3ef-419a-aaf6-c0a0da0df740" containerName="revoke-keys" Jul 14 14:07:48.845391 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.845361 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="febca3b0-b3ef-419a-aaf6-c0a0da0df740" containerName="revoke-keys" Jul 14 14:07:48.848937 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.848919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:48.851772 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.851745 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:07:48.851891 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.851780 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:07:48.856622 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.856595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq"] Jul 14 14:07:48.971686 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:48.971644 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq\" (UID: \"fe095585-68fd-435c-b160-44143e102894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:49.073160 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:49.073124 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq\" (UID: \"fe095585-68fd-435c-b160-44143e102894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:49.073720 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:49.073701 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq\" (UID: \"fe095585-68fd-435c-b160-44143e102894\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:49.161411 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:49.161328 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:49.298851 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:49.298827 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq"] Jul 14 14:07:49.301266 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:07:49.301235 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe095585_68fd_435c_b160_44143e102894.slice/crio-8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c WatchSource:0}: Error finding container 8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c: Status 404 returned error can't find the container with id 8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c Jul 14 14:07:49.356410 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:49.356375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" event={"ID":"fe095585-68fd-435c-b160-44143e102894","Type":"ContainerStarted","Data":"8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c"} Jul 14 14:07:50.363174 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:50.363137 2568 generic.go:356] "Generic (PLEG): container finished" podID="fe095585-68fd-435c-b160-44143e102894" containerID="e094621bc3fdd77168f02ad314457c8caad924cb36ee3cba845593f192a67b59" exitCode=0 Jul 14 14:07:50.363553 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:50.363229 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" event={"ID":"fe095585-68fd-435c-b160-44143e102894","Type":"ContainerDied","Data":"e094621bc3fdd77168f02ad314457c8caad924cb36ee3cba845593f192a67b59"} Jul 14 14:07:51.500283 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:51.500253 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:51.594555 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:51.594515 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca\") pod \"fe095585-68fd-435c-b160-44143e102894\" (UID: \"fe095585-68fd-435c-b160-44143e102894\") " Jul 14 14:07:51.594875 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:51.594853 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fe095585-68fd-435c-b160-44143e102894" (UID: "fe095585-68fd-435c-b160-44143e102894"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:07:51.696263 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:51.696166 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe095585-68fd-435c-b160-44143e102894-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:07:52.374541 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:52.374509 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" Jul 14 14:07:52.374541 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:52.374530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq" event={"ID":"fe095585-68fd-435c-b160-44143e102894","Type":"ContainerDied","Data":"8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c"} Jul 14 14:07:52.374775 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:07:52.374579 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8429693af6f77c0a080f7783312a649d57d0b9a3b821ae0251cf228ca933fa2c" Jul 14 14:08:02.296109 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:08:02.296073 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:08:23.486719 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:08:23.486674 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:08:40.097594 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:08:40.097554 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:09:00.186950 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:09:00.186911 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:09:26.186582 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:09:26.186546 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:09:51.079664 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:09:51.079630 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:10:08.621846 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:10:08.621815 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:10:26.087744 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:10:26.087703 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:10:43.781378 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:10:43.781297 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:10:58.290037 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:10:58.290001 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:11:14.793231 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:11:14.793198 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:11:33.681297 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:11:33.681256 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:11:49.691715 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:11:49.691676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:12:06.191408 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:06.191332 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:12:14.251935 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:14.251905 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:12:14.257564 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:14.257540 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:12:14.259270 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:14.259251 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:12:14.264553 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:14.264534 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b6mvv_745ddaf3-baf3-4b8a-b12b-cea1b04446e7/ovn-acl-logging/0.log" Jul 14 14:12:21.478103 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:21.478070 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:12:41.585612 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:41.585579 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:12:57.274039 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:12:57.274006 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:13:15.581524 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:15.581490 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:13:30.485817 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:30.485783 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:13:46.577126 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:46.577090 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:13:56.088345 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.088309 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7790-139626453263152"] Jul 14 14:13:56.088724 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.088714 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe095585-68fd-435c-b160-44143e102894" containerName="revoke-keys" Jul 14 14:13:56.088724 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.088738 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe095585-68fd-435c-b160-44143e102894" containerName="revoke-keys" Jul 14 14:13:56.088837 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.088826 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe095585-68fd-435c-b160-44143e102894" containerName="revoke-keys" Jul 14 14:13:56.091909 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.091891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:56.095268 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.095247 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-t7lm6\"" Jul 14 14:13:56.102547 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.102523 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453263152"] Jul 14 14:13:56.163708 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.163668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs7mh\" (UniqueName: \"kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh\") pod \"test-curl-7790-139626453263152\" (UID: \"15a57934-4917-47ee-8849-a46a7c028776\") " pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:56.264373 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.264328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs7mh\" (UniqueName: \"kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh\") pod \"test-curl-7790-139626453263152\" (UID: \"15a57934-4917-47ee-8849-a46a7c028776\") " pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:56.274667 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.274631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs7mh\" (UniqueName: \"kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh\") pod \"test-curl-7790-139626453263152\" (UID: \"15a57934-4917-47ee-8849-a46a7c028776\") " pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:56.401896 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.401812 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:56.540670 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.540645 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453263152"] Jul 14 14:13:56.546187 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:56.546159 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 14:13:57.324816 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:57.324771 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453263152" event={"ID":"15a57934-4917-47ee-8849-a46a7c028776","Type":"ContainerStarted","Data":"db8bd2ce370eed66db2c0eaa3a7b98a359d11b0e67f582a5412630c7c7131dcb"} Jul 14 14:13:57.325212 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:57.324834 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453263152" event={"ID":"15a57934-4917-47ee-8849-a46a7c028776","Type":"ContainerStarted","Data":"c66ae0dfa7f1e6d4b975376397904a6684ab5831edbaf8cff5085a8d0e992255"} Jul 14 14:13:57.458994 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:57.458963 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7790-139626453263152_15a57934-4917-47ee-8849-a46a7c028776/test-curl-7790-139626453263152/0.log" Jul 14 14:13:58.332358 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:58.332325 2568 generic.go:356] "Generic (PLEG): container finished" podID="15a57934-4917-47ee-8849-a46a7c028776" containerID="db8bd2ce370eed66db2c0eaa3a7b98a359d11b0e67f582a5412630c7c7131dcb" exitCode=0 Jul 14 14:13:58.332358 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:58.332364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453263152" event={"ID":"15a57934-4917-47ee-8849-a46a7c028776","Type":"ContainerDied","Data":"db8bd2ce370eed66db2c0eaa3a7b98a359d11b0e67f582a5412630c7c7131dcb"} Jul 14 14:13:59.476127 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.476099 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:13:59.494943 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.494913 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs7mh\" (UniqueName: \"kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh\") pod \"15a57934-4917-47ee-8849-a46a7c028776\" (UID: \"15a57934-4917-47ee-8849-a46a7c028776\") " Jul 14 14:13:59.497290 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.497255 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh" (OuterVolumeSpecName: "kube-api-access-fs7mh") pod "15a57934-4917-47ee-8849-a46a7c028776" (UID: "15a57934-4917-47ee-8849-a46a7c028776"). InnerVolumeSpecName "kube-api-access-fs7mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:13:59.511674 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.511644 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7790-139626453263152"] Jul 14 14:13:59.518212 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.518185 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7790-139626453263152"] Jul 14 14:13:59.596026 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.595931 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs7mh\" (UniqueName: \"kubernetes.io/projected/15a57934-4917-47ee-8849-a46a7c028776-kube-api-access-fs7mh\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:13:59.658611 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.658576 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7790-139626453237808"] Jul 14 14:13:59.658982 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.658968 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15a57934-4917-47ee-8849-a46a7c028776" containerName="test-curl-7790-139626453263152" Jul 14 14:13:59.659033 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.658983 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="15a57934-4917-47ee-8849-a46a7c028776" containerName="test-curl-7790-139626453263152" Jul 14 14:13:59.659068 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.659054 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="15a57934-4917-47ee-8849-a46a7c028776" containerName="test-curl-7790-139626453263152" Jul 14 14:13:59.662146 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.662125 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:13:59.670657 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.670631 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453237808"] Jul 14 14:13:59.696583 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.696544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmk6p\" (UniqueName: \"kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p\") pod \"test-curl-7790-139626453237808\" (UID: \"309d80f5-55d8-429b-b5b3-e429ecf9f9bf\") " pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:13:59.797674 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.797641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmk6p\" (UniqueName: \"kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p\") pod \"test-curl-7790-139626453237808\" (UID: \"309d80f5-55d8-429b-b5b3-e429ecf9f9bf\") " pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:13:59.808027 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.807996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmk6p\" (UniqueName: \"kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p\") pod \"test-curl-7790-139626453237808\" (UID: \"309d80f5-55d8-429b-b5b3-e429ecf9f9bf\") " pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:13:59.972247 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:13:59.972156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:14:00.109173 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.109141 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453237808"] Jul 14 14:14:00.110975 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:00.110947 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod309d80f5_55d8_429b_b5b3_e429ecf9f9bf.slice/crio-69d546c743abdeef5a49ce7653c68b6fe88ed933afbe71f5efd57e32090aed57 WatchSource:0}: Error finding container 69d546c743abdeef5a49ce7653c68b6fe88ed933afbe71f5efd57e32090aed57: Status 404 returned error can't find the container with id 69d546c743abdeef5a49ce7653c68b6fe88ed933afbe71f5efd57e32090aed57 Jul 14 14:14:00.138226 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.138193 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15a57934-4917-47ee-8849-a46a7c028776" path="/var/lib/kubelet/pods/15a57934-4917-47ee-8849-a46a7c028776/volumes" Jul 14 14:14:00.343641 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.343610 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453263152" Jul 14 14:14:00.343860 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.343666 2568 scope.go:117] "RemoveContainer" containerID="db8bd2ce370eed66db2c0eaa3a7b98a359d11b0e67f582a5412630c7c7131dcb" Jul 14 14:14:00.344937 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.344903 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453237808" event={"ID":"309d80f5-55d8-429b-b5b3-e429ecf9f9bf","Type":"ContainerStarted","Data":"69d546c743abdeef5a49ce7653c68b6fe88ed933afbe71f5efd57e32090aed57"} Jul 14 14:14:00.687966 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:00.687883 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:14:01.352886 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:01.352853 2568 generic.go:356] "Generic (PLEG): container finished" podID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" containerID="ce91464d1b694eee50467022407a9c2b1fbb9181a449206fed91a0f07fab0c88" exitCode=0 Jul 14 14:14:01.353063 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:01.352891 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453237808" event={"ID":"309d80f5-55d8-429b-b5b3-e429ecf9f9bf","Type":"ContainerDied","Data":"ce91464d1b694eee50467022407a9c2b1fbb9181a449206fed91a0f07fab0c88"} Jul 14 14:14:02.507500 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.507469 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:14:02.534100 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.534076 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7790-139626453237808_309d80f5-55d8-429b-b5b3-e429ecf9f9bf/test-curl-7790-139626453237808/0.log" Jul 14 14:14:02.562546 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.562505 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7790-139626453237808"] Jul 14 14:14:02.569223 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.569194 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7790-139626453237808"] Jul 14 14:14:02.623093 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.623000 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmk6p\" (UniqueName: \"kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p\") pod \"309d80f5-55d8-429b-b5b3-e429ecf9f9bf\" (UID: \"309d80f5-55d8-429b-b5b3-e429ecf9f9bf\") " Jul 14 14:14:02.625046 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.625018 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p" (OuterVolumeSpecName: "kube-api-access-nmk6p") pod "309d80f5-55d8-429b-b5b3-e429ecf9f9bf" (UID: "309d80f5-55d8-429b-b5b3-e429ecf9f9bf"). InnerVolumeSpecName "kube-api-access-nmk6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:02.724397 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.724361 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmk6p\" (UniqueName: \"kubernetes.io/projected/309d80f5-55d8-429b-b5b3-e429ecf9f9bf-kube-api-access-nmk6p\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:02.733804 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.733772 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:02.734233 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.734214 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" containerName="test-curl-7790-139626453237808" Jul 14 14:14:02.734320 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.734235 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" containerName="test-curl-7790-139626453237808" Jul 14 14:14:02.734390 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.734332 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" containerName="test-curl-7790-139626453237808" Jul 14 14:14:02.737438 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.737415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:02.744638 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.744578 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:02.825322 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.825278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69txp\" (UniqueName: \"kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp\") pod \"test-curl-7790-139626453714672\" (UID: \"1761911d-8a2f-404f-8606-c2f8114e0154\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:02.926367 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.926272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69txp\" (UniqueName: \"kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp\") pod \"test-curl-7790-139626453714672\" (UID: \"1761911d-8a2f-404f-8606-c2f8114e0154\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:02.936352 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:02.936319 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69txp\" (UniqueName: \"kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp\") pod \"test-curl-7790-139626453714672\" (UID: \"1761911d-8a2f-404f-8606-c2f8114e0154\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:03.047165 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:03.047129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:03.184949 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:03.184865 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:03.187785 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:03.187756 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1761911d_8a2f_404f_8606_c2f8114e0154.slice/crio-4ea24ef636995e2e3bdba953f2cea79c72ebf8bf7a15a7e6167cd6db370cf053 WatchSource:0}: Error finding container 4ea24ef636995e2e3bdba953f2cea79c72ebf8bf7a15a7e6167cd6db370cf053: Status 404 returned error can't find the container with id 4ea24ef636995e2e3bdba953f2cea79c72ebf8bf7a15a7e6167cd6db370cf053 Jul 14 14:14:03.364258 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:03.364213 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453714672" event={"ID":"1761911d-8a2f-404f-8606-c2f8114e0154","Type":"ContainerStarted","Data":"4ea24ef636995e2e3bdba953f2cea79c72ebf8bf7a15a7e6167cd6db370cf053"} Jul 14 14:14:03.365351 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:03.365332 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:14:03.365351 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:03.365343 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69d546c743abdeef5a49ce7653c68b6fe88ed933afbe71f5efd57e32090aed57" Jul 14 14:14:03.375968 ip-10-0-138-114 kubenswrapper[2568]: E0714 14:14:03.375944 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7790-139626453237808\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:14:04.137823 ip-10-0-138-114 kubenswrapper[2568]: E0714 14:14:04.137779 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7790-139626453237808\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" pod="openshift-ingress/test-curl-7790-139626453237808" Jul 14 14:14:04.138598 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:04.138575 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309d80f5-55d8-429b-b5b3-e429ecf9f9bf" path="/var/lib/kubelet/pods/309d80f5-55d8-429b-b5b3-e429ecf9f9bf/volumes" Jul 14 14:14:04.372047 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:04.372016 2568 generic.go:356] "Generic (PLEG): container finished" podID="1761911d-8a2f-404f-8606-c2f8114e0154" containerID="7c782f78ee057ff0dfd4f0a0a2265ed48818f530ac22852c24fe156e7e43fd5b" exitCode=0 Jul 14 14:14:04.372219 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:04.372060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453714672" event={"ID":"1761911d-8a2f-404f-8606-c2f8114e0154","Type":"ContainerDied","Data":"7c782f78ee057ff0dfd4f0a0a2265ed48818f530ac22852c24fe156e7e43fd5b"} Jul 14 14:14:05.517544 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.517513 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:05.541752 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.541706 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7790-139626453714672_1761911d-8a2f-404f-8606-c2f8114e0154/test-curl-7790-139626453714672/0.log" Jul 14 14:14:05.548957 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.548932 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69txp\" (UniqueName: \"kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp\") pod \"1761911d-8a2f-404f-8606-c2f8114e0154\" (UID: \"1761911d-8a2f-404f-8606-c2f8114e0154\") " Jul 14 14:14:05.551007 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.550985 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp" (OuterVolumeSpecName: "kube-api-access-69txp") pod "1761911d-8a2f-404f-8606-c2f8114e0154" (UID: "1761911d-8a2f-404f-8606-c2f8114e0154"). InnerVolumeSpecName "kube-api-access-69txp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:05.569053 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.569010 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:05.571380 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.571354 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:05.649797 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.649678 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69txp\" (UniqueName: \"kubernetes.io/projected/1761911d-8a2f-404f-8606-c2f8114e0154-kube-api-access-69txp\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:05.701075 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.701037 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:05.701419 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.701407 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1761911d-8a2f-404f-8606-c2f8114e0154" containerName="test-curl-7790-139626453714672" Jul 14 14:14:05.701459 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.701420 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1761911d-8a2f-404f-8606-c2f8114e0154" containerName="test-curl-7790-139626453714672" Jul 14 14:14:05.701514 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.701505 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1761911d-8a2f-404f-8606-c2f8114e0154" containerName="test-curl-7790-139626453714672" Jul 14 14:14:05.704432 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.704416 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:05.711283 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.711236 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:05.750412 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.750378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fl2s\" (UniqueName: \"kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s\") pod \"test-curl-7790-139626453714672\" (UID: \"c28aaf6b-f545-4048-a181-6bc857a85139\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:05.851082 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.851037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fl2s\" (UniqueName: \"kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s\") pod \"test-curl-7790-139626453714672\" (UID: \"c28aaf6b-f545-4048-a181-6bc857a85139\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:05.860719 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:05.860691 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fl2s\" (UniqueName: \"kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s\") pod \"test-curl-7790-139626453714672\" (UID: \"c28aaf6b-f545-4048-a181-6bc857a85139\") " pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:06.014560 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.014457 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:06.138590 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.138554 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1761911d-8a2f-404f-8606-c2f8114e0154" path="/var/lib/kubelet/pods/1761911d-8a2f-404f-8606-c2f8114e0154/volumes" Jul 14 14:14:06.155892 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.155866 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:06.157437 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:06.157414 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc28aaf6b_f545_4048_a181_6bc857a85139.slice/crio-3741268f9c9d336e269e999f29789d9cd3caacc1a4849225b99797efa573dfc9 WatchSource:0}: Error finding container 3741268f9c9d336e269e999f29789d9cd3caacc1a4849225b99797efa573dfc9: Status 404 returned error can't find the container with id 3741268f9c9d336e269e999f29789d9cd3caacc1a4849225b99797efa573dfc9 Jul 14 14:14:06.385288 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.385259 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:06.385288 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.385281 2568 scope.go:117] "RemoveContainer" containerID="7c782f78ee057ff0dfd4f0a0a2265ed48818f530ac22852c24fe156e7e43fd5b" Jul 14 14:14:06.386815 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:06.386787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453714672" event={"ID":"c28aaf6b-f545-4048-a181-6bc857a85139","Type":"ContainerStarted","Data":"3741268f9c9d336e269e999f29789d9cd3caacc1a4849225b99797efa573dfc9"} Jul 14 14:14:07.394475 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:07.394440 2568 generic.go:356] "Generic (PLEG): container finished" podID="c28aaf6b-f545-4048-a181-6bc857a85139" containerID="656425179f94efb63a5f6ac7bbd35903036141e12d8417c8d8761b891fcb84d0" exitCode=0 Jul 14 14:14:07.394919 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:07.394520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626453714672" event={"ID":"c28aaf6b-f545-4048-a181-6bc857a85139","Type":"ContainerDied","Data":"656425179f94efb63a5f6ac7bbd35903036141e12d8417c8d8761b891fcb84d0"} Jul 14 14:14:08.542711 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.542686 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:08.565501 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.565475 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7790-139626453714672_c28aaf6b-f545-4048-a181-6bc857a85139/test-curl-7790-139626453714672/0.log" Jul 14 14:14:08.576755 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.576698 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fl2s\" (UniqueName: \"kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s\") pod \"c28aaf6b-f545-4048-a181-6bc857a85139\" (UID: \"c28aaf6b-f545-4048-a181-6bc857a85139\") " Jul 14 14:14:08.578805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.578774 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s" (OuterVolumeSpecName: "kube-api-access-6fl2s") pod "c28aaf6b-f545-4048-a181-6bc857a85139" (UID: "c28aaf6b-f545-4048-a181-6bc857a85139"). InnerVolumeSpecName "kube-api-access-6fl2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:08.592173 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.592139 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:08.595917 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.595893 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7790-139626453714672"] Jul 14 14:14:08.677907 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.677819 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fl2s\" (UniqueName: \"kubernetes.io/projected/c28aaf6b-f545-4048-a181-6bc857a85139-kube-api-access-6fl2s\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:08.718556 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.718522 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7790-139626452215104"] Jul 14 14:14:08.718947 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.718934 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c28aaf6b-f545-4048-a181-6bc857a85139" containerName="test-curl-7790-139626453714672" Jul 14 14:14:08.718988 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.718949 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c28aaf6b-f545-4048-a181-6bc857a85139" containerName="test-curl-7790-139626453714672" Jul 14 14:14:08.719035 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.719026 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c28aaf6b-f545-4048-a181-6bc857a85139" containerName="test-curl-7790-139626453714672" Jul 14 14:14:08.722031 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.722011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:08.727409 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.727386 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626452215104"] Jul 14 14:14:08.779267 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.779219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7k4q\" (UniqueName: \"kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q\") pod \"test-curl-7790-139626452215104\" (UID: \"7a2b7c05-06b6-459b-bf39-ea21591958ae\") " pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:08.880052 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.880010 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7k4q\" (UniqueName: \"kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q\") pod \"test-curl-7790-139626452215104\" (UID: \"7a2b7c05-06b6-459b-bf39-ea21591958ae\") " pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:08.889570 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:08.889535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7k4q\" (UniqueName: \"kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q\") pod \"test-curl-7790-139626452215104\" (UID: \"7a2b7c05-06b6-459b-bf39-ea21591958ae\") " pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:09.032870 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:09.032837 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:09.169175 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:09.169145 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7790-139626452215104"] Jul 14 14:14:09.170182 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:09.170158 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2b7c05_06b6_459b_bf39_ea21591958ae.slice/crio-745401cfe595b6f0da6540c7534fb8317c7c254b8fb98668b77757646fe522ec WatchSource:0}: Error finding container 745401cfe595b6f0da6540c7534fb8317c7c254b8fb98668b77757646fe522ec: Status 404 returned error can't find the container with id 745401cfe595b6f0da6540c7534fb8317c7c254b8fb98668b77757646fe522ec Jul 14 14:14:09.409357 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:09.409276 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:09.409357 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:09.409309 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3741268f9c9d336e269e999f29789d9cd3caacc1a4849225b99797efa573dfc9" Jul 14 14:14:09.410501 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:09.410477 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626452215104" event={"ID":"7a2b7c05-06b6-459b-bf39-ea21591958ae","Type":"ContainerStarted","Data":"745401cfe595b6f0da6540c7534fb8317c7c254b8fb98668b77757646fe522ec"} Jul 14 14:14:09.420266 ip-10-0-138-114 kubenswrapper[2568]: E0714 14:14:09.420237 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7790-139626453714672\" is forbidden: User \"system:node:ip-10-0-138-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-114.ec2.internal' and this object" podUID="c28aaf6b-f545-4048-a181-6bc857a85139" pod="openshift-ingress/test-curl-7790-139626453714672" Jul 14 14:14:10.138346 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:10.138309 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c28aaf6b-f545-4048-a181-6bc857a85139" path="/var/lib/kubelet/pods/c28aaf6b-f545-4048-a181-6bc857a85139/volumes" Jul 14 14:14:10.418402 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:10.418307 2568 generic.go:356] "Generic (PLEG): container finished" podID="7a2b7c05-06b6-459b-bf39-ea21591958ae" containerID="992b90bcf7ad59f187a9f1f21de62edb230340c32cde89090ea2e43b903d04a4" exitCode=0 Jul 14 14:14:10.418572 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:10.418394 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7790-139626452215104" event={"ID":"7a2b7c05-06b6-459b-bf39-ea21591958ae","Type":"ContainerDied","Data":"992b90bcf7ad59f187a9f1f21de62edb230340c32cde89090ea2e43b903d04a4"} Jul 14 14:14:11.575146 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.575121 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:11.599759 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.599719 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7790-139626452215104_7a2b7c05-06b6-459b-bf39-ea21591958ae/test-curl-7790-139626452215104/0.log" Jul 14 14:14:11.625275 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.625242 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7790-139626452215104"] Jul 14 14:14:11.632226 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.632199 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7790-139626452215104"] Jul 14 14:14:11.702707 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.702601 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7k4q\" (UniqueName: \"kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q\") pod \"7a2b7c05-06b6-459b-bf39-ea21591958ae\" (UID: \"7a2b7c05-06b6-459b-bf39-ea21591958ae\") " Jul 14 14:14:11.704787 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.704755 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q" (OuterVolumeSpecName: "kube-api-access-f7k4q") pod "7a2b7c05-06b6-459b-bf39-ea21591958ae" (UID: "7a2b7c05-06b6-459b-bf39-ea21591958ae"). InnerVolumeSpecName "kube-api-access-f7k4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:14:11.803514 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:11.803478 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7k4q\" (UniqueName: \"kubernetes.io/projected/7a2b7c05-06b6-459b-bf39-ea21591958ae-kube-api-access-f7k4q\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:12.138096 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:12.138062 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2b7c05-06b6-459b-bf39-ea21591958ae" path="/var/lib/kubelet/pods/7a2b7c05-06b6-459b-bf39-ea21591958ae/volumes" Jul 14 14:14:12.429902 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:12.429801 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7790-139626452215104" Jul 14 14:14:12.429902 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:12.429845 2568 scope.go:117] "RemoveContainer" containerID="992b90bcf7ad59f187a9f1f21de62edb230340c32cde89090ea2e43b903d04a4" Jul 14 14:14:15.365502 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.360613 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s"] Jul 14 14:14:15.365502 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.361483 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2b7c05-06b6-459b-bf39-ea21591958ae" containerName="test-curl-7790-139626452215104" Jul 14 14:14:15.365502 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.361506 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2b7c05-06b6-459b-bf39-ea21591958ae" containerName="test-curl-7790-139626452215104" Jul 14 14:14:15.365502 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.361682 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2b7c05-06b6-459b-bf39-ea21591958ae" containerName="test-curl-7790-139626452215104" Jul 14 14:14:15.367541 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.367494 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:15.370472 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.370447 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:14:15.370472 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.370464 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:14:15.371555 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.371532 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s"] Jul 14 14:14:15.438567 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.438527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s\" (UID: \"66712e6f-0f9b-4d17-8409-9558a2d43b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:15.539953 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.539904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s\" (UID: \"66712e6f-0f9b-4d17-8409-9558a2d43b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:15.540572 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.540552 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s\" (UID: \"66712e6f-0f9b-4d17-8409-9558a2d43b61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:15.680296 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.680204 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:15.823960 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:15.823935 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s"] Jul 14 14:14:15.825950 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:15.825913 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66712e6f_0f9b_4d17_8409_9558a2d43b61.slice/crio-055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5 WatchSource:0}: Error finding container 055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5: Status 404 returned error can't find the container with id 055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5 Jul 14 14:14:16.454789 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:16.454751 2568 generic.go:356] "Generic (PLEG): container finished" podID="66712e6f-0f9b-4d17-8409-9558a2d43b61" containerID="335aea64968bd6ccc06b95ad81e9af0941cff1967e06912428c464fa4ec98794" exitCode=0 Jul 14 14:14:16.455197 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:16.454837 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" event={"ID":"66712e6f-0f9b-4d17-8409-9558a2d43b61","Type":"ContainerDied","Data":"335aea64968bd6ccc06b95ad81e9af0941cff1967e06912428c464fa4ec98794"} Jul 14 14:14:16.455197 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:16.454901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" event={"ID":"66712e6f-0f9b-4d17-8409-9558a2d43b61","Type":"ContainerStarted","Data":"055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5"} Jul 14 14:14:17.598665 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:17.598636 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:17.661673 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:17.661636 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca\") pod \"66712e6f-0f9b-4d17-8409-9558a2d43b61\" (UID: \"66712e6f-0f9b-4d17-8409-9558a2d43b61\") " Jul 14 14:14:17.662013 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:17.661988 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66712e6f-0f9b-4d17-8409-9558a2d43b61" (UID: "66712e6f-0f9b-4d17-8409-9558a2d43b61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:14:17.763011 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:17.762912 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66712e6f-0f9b-4d17-8409-9558a2d43b61-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:18.466585 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:18.466506 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" Jul 14 14:14:18.466585 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:18.466547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s" event={"ID":"66712e6f-0f9b-4d17-8409-9558a2d43b61","Type":"ContainerDied","Data":"055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5"} Jul 14 14:14:18.466786 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:18.466590 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="055e93ce6457e2fcb31c3aa3a7e815c30071d0febc1e12c15553ec581c4f3ad5" Jul 14 14:14:29.279427 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:29.279394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:14:41.569407 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.569372 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw"] Jul 14 14:14:41.569805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.569780 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66712e6f-0f9b-4d17-8409-9558a2d43b61" containerName="revoke-keys" Jul 14 14:14:41.569805 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.569792 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66712e6f-0f9b-4d17-8409-9558a2d43b61" containerName="revoke-keys" Jul 14 14:14:41.569893 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.569865 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66712e6f-0f9b-4d17-8409-9558a2d43b61" containerName="revoke-keys" Jul 14 14:14:41.574800 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.574783 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:41.577830 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.577811 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:14:41.577830 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.577818 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:14:41.580307 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.580282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw\" (UID: \"54b9ca47-ac7c-46b6-8bee-eeda65653124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:41.585854 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.585828 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw"] Jul 14 14:14:41.681256 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.681216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw\" (UID: \"54b9ca47-ac7c-46b6-8bee-eeda65653124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:41.681956 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.681933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw\" (UID: \"54b9ca47-ac7c-46b6-8bee-eeda65653124\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:41.885857 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:41.885762 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:42.036683 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:42.036655 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw"] Jul 14 14:14:42.038982 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:14:42.038947 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b9ca47_ac7c_46b6_8bee_eeda65653124.slice/crio-9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8 WatchSource:0}: Error finding container 9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8: Status 404 returned error can't find the container with id 9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8 Jul 14 14:14:42.597492 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:42.597452 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" event={"ID":"54b9ca47-ac7c-46b6-8bee-eeda65653124","Type":"ContainerStarted","Data":"caa5433cd5016e9c7eaf52719d87652112ff722b731b4aef3c9ef6ab4eb27aa3"} Jul 14 14:14:42.597911 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:42.597513 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" event={"ID":"54b9ca47-ac7c-46b6-8bee-eeda65653124","Type":"ContainerStarted","Data":"9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8"} Jul 14 14:14:42.616641 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:42.616590 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" podStartSLOduration=1.616573026 podStartE2EDuration="1.616573026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:14:42.613941566 +0000 UTC m=+2849.080069130" watchObservedRunningTime="2026-07-14 14:14:42.616573026 +0000 UTC m=+2849.082700580" Jul 14 14:14:43.604512 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:43.604475 2568 generic.go:356] "Generic (PLEG): container finished" podID="54b9ca47-ac7c-46b6-8bee-eeda65653124" containerID="caa5433cd5016e9c7eaf52719d87652112ff722b731b4aef3c9ef6ab4eb27aa3" exitCode=0 Jul 14 14:14:43.604909 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:43.604564 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" event={"ID":"54b9ca47-ac7c-46b6-8bee-eeda65653124","Type":"ContainerDied","Data":"caa5433cd5016e9c7eaf52719d87652112ff722b731b4aef3c9ef6ab4eb27aa3"} Jul 14 14:14:44.749426 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:44.749404 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:44.805615 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:44.805578 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca\") pod \"54b9ca47-ac7c-46b6-8bee-eeda65653124\" (UID: \"54b9ca47-ac7c-46b6-8bee-eeda65653124\") " Jul 14 14:14:44.806010 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:44.805985 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "54b9ca47-ac7c-46b6-8bee-eeda65653124" (UID: "54b9ca47-ac7c-46b6-8bee-eeda65653124"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:14:44.907068 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:44.906982 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/54b9ca47-ac7c-46b6-8bee-eeda65653124-maas-api-service-ca\") on node \"ip-10-0-138-114.ec2.internal\" DevicePath \"\"" Jul 14 14:14:45.616501 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:45.616467 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" event={"ID":"54b9ca47-ac7c-46b6-8bee-eeda65653124","Type":"ContainerDied","Data":"9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8"} Jul 14 14:14:45.616501 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:45.616487 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw" Jul 14 14:14:45.616747 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:45.616526 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d7c584a2179855c445e7e8858174d2c8ed187aa6ca3289e4ec8cba94d372be8" Jul 14 14:14:54.981523 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:14:54.981487 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-76nk5"] Jul 14 14:15:00.144527 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.144492 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds"] Jul 14 14:15:00.144919 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.144865 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54b9ca47-ac7c-46b6-8bee-eeda65653124" containerName="revoke-keys" Jul 14 14:15:00.144919 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.144876 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="54b9ca47-ac7c-46b6-8bee-eeda65653124" containerName="revoke-keys" Jul 14 14:15:00.144999 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.144959 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="54b9ca47-ac7c-46b6-8bee-eeda65653124" containerName="revoke-keys" Jul 14 14:15:00.148087 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.148068 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" Jul 14 14:15:00.150818 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.150797 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5w5wd\"" Jul 14 14:15:00.155892 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.155860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds"] Jul 14 14:15:00.159604 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.159579 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" Jul 14 14:15:00.300367 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.300344 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds"] Jul 14 14:15:00.302166 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:15:00.302142 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a73ef34_49d9_45f2_824f_960ef16841a1.slice/crio-0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e WatchSource:0}: Error finding container 0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e: Status 404 returned error can't find the container with id 0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e Jul 14 14:15:00.699216 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.699181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" event={"ID":"2a73ef34-49d9-45f2-824f-960ef16841a1","Type":"ContainerStarted","Data":"d9a5903a38d22d16011a5e1030f943e9b59e1a823d91bc15e366f6f1ae8c6008"} Jul 14 14:15:00.699388 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.699232 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" event={"ID":"2a73ef34-49d9-45f2-824f-960ef16841a1","Type":"ContainerStarted","Data":"0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e"} Jul 14 14:15:00.725093 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:00.725047 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" podStartSLOduration=0.72503176 podStartE2EDuration="725.03176ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:15:00.723152436 +0000 UTC m=+2867.189279989" watchObservedRunningTime="2026-07-14 14:15:00.72503176 +0000 UTC m=+2867.191159314" Jul 14 14:15:01.705854 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:01.705813 2568 generic.go:356] "Generic (PLEG): container finished" podID="2a73ef34-49d9-45f2-824f-960ef16841a1" containerID="d9a5903a38d22d16011a5e1030f943e9b59e1a823d91bc15e366f6f1ae8c6008" exitCode=0 Jul 14 14:15:01.706257 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:01.705900 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" event={"ID":"2a73ef34-49d9-45f2-824f-960ef16841a1","Type":"ContainerDied","Data":"d9a5903a38d22d16011a5e1030f943e9b59e1a823d91bc15e366f6f1ae8c6008"} Jul 14 14:15:02.853296 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:02.853270 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" Jul 14 14:15:03.718708 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:03.718680 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" Jul 14 14:15:03.718899 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:03.718677 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-m98ds" event={"ID":"2a73ef34-49d9-45f2-824f-960ef16841a1","Type":"ContainerDied","Data":"0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e"} Jul 14 14:15:03.718899 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:03.718810 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f42ad0ab243bd2b3ddb5151e4e209c7d7ae809c2f2beef7af1e095b19428d1e" Jul 14 14:15:13.758796 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:13.758710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-64kps_b654cbd9-ba91-400b-9d79-13fbd6f324e6/copy-manifests/0.log" Jul 14 14:15:13.766814 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:13.766787 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-64kps_b654cbd9-ba91-400b-9d79-13fbd6f324e6/manager/0.log" Jul 14 14:15:14.540534 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:14.540503 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733945-x82tt_b40b268c-02f6-422c-9b66-93c842f35095/cleanup/0.log" Jul 14 14:15:14.658364 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:14.658337 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733960-c52gv_0ca454f8-d8d6-4378-852a-4dbb7f884106/cleanup/0.log" Jul 14 14:15:14.784225 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:14.784196 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733975-m98ds_2a73ef34-49d9-45f2-824f-960ef16841a1/cleanup/0.log" Jul 14 14:15:14.903272 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:14.903191 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b0840d76-7d9qg_1e918f85-d93f-4047-ae4d-20586e7fa1ae/revoke-keys/0.log" Jul 14 14:15:15.033643 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.033613 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-b3b6f94a-qh7jq_55e4606c-7066-4f9f-8419-dce730c990ee/revoke-keys/0.log" Jul 14 14:15:15.170598 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.170522 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-f2a083-vqq65_febca3b0-b3ef-419a-aaf6-c0a0da0df740/revoke-keys/0.log" Jul 14 14:15:15.295335 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.295305 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-1816dc-dvfxq_fe095585-68fd-435c-b160-44143e102894/revoke-keys/0.log" Jul 14 14:15:15.405528 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.405498 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-fe2b25e2-2nrdx_7ae82ba5-e5e9-46bf-97b2-f7e80bf8f6e8/revoke-keys/0.log" Jul 14 14:15:15.522060 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.521979 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-370ee8-vlksj_33f19320-1639-48e1-b9dd-e8ce407dc5bd/revoke-keys/0.log" Jul 14 14:15:15.646592 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.646559 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-f1dea9-qlmjd_39174521-7e2e-4604-a56d-4a2e7789e1c3/revoke-keys/0.log" Jul 14 14:15:15.759963 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.759932 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-0df6d38a-g22cm_a37ab69b-b0a6-4c2b-8a7b-4d6c052d2f00/revoke-keys/0.log" Jul 14 14:15:15.875854 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.875827 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-4ca3631f-zb9fk_32c8756d-24f3-499f-b01f-25f9ee281d67/revoke-keys/0.log" Jul 14 14:15:15.991239 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:15.991209 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-50aeb779-w79mh_84a814ef-89aa-4545-85a4-b9a77520f4f3/revoke-keys/0.log" Jul 14 14:15:16.104665 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:16.104633 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-9c0e14-2qm2s_66712e6f-0f9b-4d17-8409-9558a2d43b61/revoke-keys/0.log" Jul 14 14:15:16.219907 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:16.219827 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-ff9f03-d59qw_54b9ca47-ac7c-46b6-8bee-eeda65653124/revoke-keys/0.log" Jul 14 14:15:18.481739 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:18.481710 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-76nk5_90a78eef-508c-4e3d-a745-e69dc89dbda1/limitador/0.log" Jul 14 14:15:19.152277 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:19.152246 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ndh45_e43167fe-b827-4bce-b62a-e991cdd27eb1/discovery/0.log" Jul 14 14:15:19.900122 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:19.900097 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7b478588-9cfpm_93b1c003-057f-48ca-abbb-e1f6b4b3bbc9/router/0.log" Jul 14 14:15:20.639246 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:20.639214 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbt285x_d0455cd9-40de-4079-a783-f2617faeb385/main/0.log" Jul 14 14:15:28.162014 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:28.161984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-krb8t_d139e885-1141-43fc-a857-62e336ce02c5/global-pull-secret-syncer/0.log" Jul 14 14:15:28.381522 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:28.381488 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r5cbh_9cfe8616-0e6e-43a0-86f3-74dbdaf03607/konnectivity-agent/0.log" Jul 14 14:15:28.404512 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:28.404485 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-114.ec2.internal_c6d51354587a329354b548be5d6cdd37/haproxy/0.log" Jul 14 14:15:33.595701 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:33.595671 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-76nk5_90a78eef-508c-4e3d-a745-e69dc89dbda1/limitador/0.log" Jul 14 14:15:35.417194 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.417168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/alertmanager/0.log" Jul 14 14:15:35.441722 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.441696 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/config-reloader/0.log" Jul 14 14:15:35.464895 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.464868 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/kube-rbac-proxy-web/0.log" Jul 14 14:15:35.495119 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.495096 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/kube-rbac-proxy/0.log" Jul 14 14:15:35.522814 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.522792 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/kube-rbac-proxy-metric/0.log" Jul 14 14:15:35.547529 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.547506 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/prom-label-proxy/0.log" Jul 14 14:15:35.579914 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.579884 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9d1c1411-2425-4548-996e-60b2a89b10e8/init-config-reloader/0.log" Jul 14 14:15:35.636020 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.635984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-zj5sx_c31b1f5b-825c-4836-ba9f-ba304b7d4c45/cluster-monitoring-operator/0.log" Jul 14 14:15:35.777690 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.777675 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85f8b5cb44-lhjkl_5569d118-8f6d-40c5-8fc9-7a5e563987fd/metrics-server/0.log" Jul 14 14:15:35.808712 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.808684 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-wtd5g_1ce5a003-11b9-4045-91a2-6991a504d8aa/monitoring-plugin/0.log" Jul 14 14:15:35.920833 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.920804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwc8k_97294527-1356-4ac2-89ce-f05db1255f12/node-exporter/0.log" Jul 14 14:15:35.946831 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.946807 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwc8k_97294527-1356-4ac2-89ce-f05db1255f12/kube-rbac-proxy/0.log" Jul 14 14:15:35.970960 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:35.970940 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nwc8k_97294527-1356-4ac2-89ce-f05db1255f12/init-textfile/0.log" Jul 14 14:15:36.079600 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.079520 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-czcl6_c3613939-6924-408d-af40-84bc02cd7c43/kube-rbac-proxy-main/0.log" Jul 14 14:15:36.105044 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.105014 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-czcl6_c3613939-6924-408d-af40-84bc02cd7c43/kube-rbac-proxy-self/0.log" Jul 14 14:15:36.139623 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.139598 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-czcl6_c3613939-6924-408d-af40-84bc02cd7c43/openshift-state-metrics/0.log" Jul 14 14:15:36.421739 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.421650 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-bsf6m_5e7efe89-0f8c-46e9-93b7-8413a6028a5f/prometheus-operator-admission-webhook/0.log" Jul 14 14:15:36.535899 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.535869 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/thanos-query/0.log" Jul 14 14:15:36.562573 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.562548 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/kube-rbac-proxy-web/0.log" Jul 14 14:15:36.588059 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.588033 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/kube-rbac-proxy/0.log" Jul 14 14:15:36.615176 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.615153 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/prom-label-proxy/0.log" Jul 14 14:15:36.644028 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.643990 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/kube-rbac-proxy-rules/0.log" Jul 14 14:15:36.671020 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:36.670995 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c654f798b-8fmtg_a185f3ef-c1a2-40c3-b0cc-760931805d06/kube-rbac-proxy-metrics/0.log" Jul 14 14:15:38.124766 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.124706 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z"] Jul 14 14:15:38.125307 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.125282 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a73ef34-49d9-45f2-824f-960ef16841a1" containerName="cleanup" Jul 14 14:15:38.125307 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.125308 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a73ef34-49d9-45f2-824f-960ef16841a1" containerName="cleanup" Jul 14 14:15:38.125442 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.125428 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a73ef34-49d9-45f2-824f-960ef16841a1" containerName="cleanup" Jul 14 14:15:38.129122 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.129096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.131942 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.131915 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t9pjq\"/\"openshift-service-ca.crt\"" Jul 14 14:15:38.132085 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.132023 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t9pjq\"/\"kube-root-ca.crt\"" Jul 14 14:15:38.132156 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.132023 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t9pjq\"/\"default-dockercfg-gqsqg\"" Jul 14 14:15:38.147595 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.147568 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z"] Jul 14 14:15:38.182265 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.182229 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-sys\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.182450 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.182338 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-proc\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.182450 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.182373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-podres\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.182450 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.182403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-lib-modules\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.182585 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.182450 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j87ql\" (UniqueName: \"kubernetes.io/projected/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-kube-api-access-j87ql\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283467 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-proc\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283467 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283466 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-podres\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-lib-modules\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j87ql\" (UniqueName: \"kubernetes.io/projected/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-kube-api-access-j87ql\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283564 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-proc\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283623 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-podres\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-lib-modules\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283709 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-sys\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.283935 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.283746 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-sys\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.293337 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.293313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j87ql\" (UniqueName: \"kubernetes.io/projected/e8c99f8d-e3ae-4b72-984d-b8942e86c3f9-kube-api-access-j87ql\") pod \"perf-node-gather-daemonset-jvd9z\" (UID: \"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9\") " pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.447547 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.447450 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.519193 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.519167 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/1.log" Jul 14 14:15:38.524520 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.524494 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-nrmmv_94a009c8-8970-4d30-b26e-44c8f404767b/console-operator/2.log" Jul 14 14:15:38.593386 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.593361 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z"] Jul 14 14:15:38.594947 ip-10-0-138-114 kubenswrapper[2568]: W0714 14:15:38.594918 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode8c99f8d_e3ae_4b72_984d_b8942e86c3f9.slice/crio-b7de1c825292da2858f3ec11ef28ff2c30a9c27dac9dfe1da9a1dc30b8abca16 WatchSource:0}: Error finding container b7de1c825292da2858f3ec11ef28ff2c30a9c27dac9dfe1da9a1dc30b8abca16: Status 404 returned error can't find the container with id b7de1c825292da2858f3ec11ef28ff2c30a9c27dac9dfe1da9a1dc30b8abca16 Jul 14 14:15:38.914871 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.914835 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" event={"ID":"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9","Type":"ContainerStarted","Data":"db0b2ab110ebf348af1b5c2b5cf9786e15bbbccc03af81f708d00e45ddbf7089"} Jul 14 14:15:38.915073 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.914887 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" event={"ID":"e8c99f8d-e3ae-4b72-984d-b8942e86c3f9","Type":"ContainerStarted","Data":"b7de1c825292da2858f3ec11ef28ff2c30a9c27dac9dfe1da9a1dc30b8abca16"} Jul 14 14:15:38.915073 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.914962 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:38.933308 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:38.933252 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" podStartSLOduration=0.933233756 podStartE2EDuration="933.233756ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 14:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:15:38.931947617 +0000 UTC m=+2905.398075172" watchObservedRunningTime="2026-07-14 14:15:38.933233756 +0000 UTC m=+2905.399361310" Jul 14 14:15:39.073352 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:39.073320 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-p6xv5_83480c3f-ad8c-484d-9661-f75f3bd7e293/download-server/0.log" Jul 14 14:15:39.685674 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:39.685645 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-btsv8_f751502f-83f6-45a5-9847-c556e5aaacd1/volume-data-source-validator/0.log" Jul 14 14:15:40.849006 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:40.848981 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ls9n8_40b314fe-62fc-4a4c-9fce-c782da4854a4/dns/0.log" Jul 14 14:15:40.872016 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:40.871972 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ls9n8_40b314fe-62fc-4a4c-9fce-c782da4854a4/kube-rbac-proxy/0.log" Jul 14 14:15:40.897964 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:40.897940 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdwtm_b30953bd-322a-4251-b5c4-1fb7799a8384/dns-node-resolver/0.log" Jul 14 14:15:41.570190 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:41.570157 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t8wfc_b4df8cf1-1b78-4cb1-a6b3-d0239d090817/node-ca/0.log" Jul 14 14:15:42.672173 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:42.672135 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ndh45_e43167fe-b827-4bce-b62a-e991cdd27eb1/discovery/0.log" Jul 14 14:15:42.959960 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:42.959870 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7b478588-9cfpm_93b1c003-057f-48ca-abbb-e1f6b4b3bbc9/router/0.log" Jul 14 14:15:43.733854 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:43.733816 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g9z48_d4cdbd15-22b6-4b58-8ddc-1f93d1323014/serve-healthcheck-canary/0.log" Jul 14 14:15:44.266519 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.266491 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-ml48v_38ec00a8-80ff-4fde-a998-e065e0169859/insights-operator/0.log" Jul 14 14:15:44.266788 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.266665 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-ml48v_38ec00a8-80ff-4fde-a998-e065e0169859/insights-operator/1.log" Jul 14 14:15:44.367027 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.366998 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lwqjt_8f731f2a-2763-4ea2-b8a3-bf991341b06c/kube-rbac-proxy/0.log" Jul 14 14:15:44.393393 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.393365 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lwqjt_8f731f2a-2763-4ea2-b8a3-bf991341b06c/exporter/0.log" Jul 14 14:15:44.424756 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.424709 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lwqjt_8f731f2a-2763-4ea2-b8a3-bf991341b06c/extractor/0.log" Jul 14 14:15:44.931880 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:44.931850 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t9pjq/perf-node-gather-daemonset-jvd9z" Jul 14 14:15:47.021240 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:47.021205 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-64kps_b654cbd9-ba91-400b-9d79-13fbd6f324e6/manager/0.log" Jul 14 14:15:47.049272 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:47.049245 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9464f545f-64kps_b654cbd9-ba91-400b-9d79-13fbd6f324e6/copy-manifests/0.log" Jul 14 14:15:47.749167 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:47.749132 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ndh45_e43167fe-b827-4bce-b62a-e991cdd27eb1/discovery/0.log" Jul 14 14:15:47.905617 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:47.905580 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7b478588-9cfpm_93b1c003-057f-48ca-abbb-e1f6b4b3bbc9/router/0.log" Jul 14 14:15:48.622190 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:48.622117 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-fc4b75864-gq4sd_f3352af1-a3ad-42e8-8fc8-3f1b98a082b1/manager/0.log" Jul 14 14:15:48.683723 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:48.683677 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-b8lg7_7aafaee6-5695-44ad-94b8-02025064d64a/openshift-lws-operator/0.log" Jul 14 14:15:49.481360 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:49.481279 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ndh45_e43167fe-b827-4bce-b62a-e991cdd27eb1/discovery/0.log" Jul 14 14:15:49.640185 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:49.640156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c7b478588-9cfpm_93b1c003-057f-48ca-abbb-e1f6b4b3bbc9/router/0.log" Jul 14 14:15:54.215810 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:54.215774 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-wdvzj_7a7b5ed4-167d-4a0c-945b-b70e878a8026/kube-storage-version-migrator-operator/1.log" Jul 14 14:15:54.216754 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:54.216712 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-wdvzj_7a7b5ed4-167d-4a0c-945b-b70e878a8026/kube-storage-version-migrator-operator/0.log" Jul 14 14:15:55.494908 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.494823 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/kube-multus-additional-cni-plugins/0.log" Jul 14 14:15:55.527683 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.527656 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/egress-router-binary-copy/0.log" Jul 14 14:15:55.563586 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.563558 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/cni-plugins/0.log" Jul 14 14:15:55.595079 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.595049 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/bond-cni-plugin/0.log" Jul 14 14:15:55.621852 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.621822 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/routeoverride-cni/0.log" Jul 14 14:15:55.646626 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.646599 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/whereabouts-cni-bincopy/0.log" Jul 14 14:15:55.670567 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.670537 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nbgng_581c0fd4-fdfb-44fd-992c-f8728532e277/whereabouts-cni/0.log" Jul 14 14:15:55.937068 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.937036 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vkdmr_0fefb8f4-1dd4-4d15-aa77-bb897fa3528c/kube-multus/0.log" Jul 14 14:15:55.970267 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.970246 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sdbxt_137eec1b-b24f-4c5d-9eb7-81601d40c532/network-metrics-daemon/0.log" Jul 14 14:15:55.996439 ip-10-0-138-114 kubenswrapper[2568]: I0714 14:15:55.996413 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sdbxt_137eec1b-b24f-4c5d-9eb7-81601d40c532/kube-rbac-proxy/0.log"